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ABSTRACT 


Originally,  the  techniques  used  by  operational  research  teams  were  borrowed 
from  other  scientific  fields.  However,  the  scope  of  problems  addressed  by 
this  new  discipline  soon  led  to  the  development  of  special  analytic  methods, 
including  such  now  familiar  terms  as  linear  programming,  game  theory, 
dynamic  programming,  queueing  theory,  and  so  forth. 

This  growth  was  accelerated  by  the  parallel  development  of  the  high-speed 
digital  computer  and  the  modern  concepts  of  solution  algorithms  and  simulation 
models.  When  computational  storage  and  speed  were  limited,  emphasis  was  on 
the  exploitation  of  a  problem's  special  structure;  as  capabilities  have 
expanded  and  costs  have  diminished,  emphasis  has  shifted  to  the  problems  of 
data  management  for  larger-scale  problems.  Efficient  data-structure  methods 
have  led  to  new  methods  for  "unsolvable"  combinatorial  problems. 

Concurrently,  there  has  been  an  explosion  in  the  theoretical  literature. 
Specialty  journals  flourish  in  the  various  methodological  specialties,  and 
conference  offerings  have  grown  to  an  unwieldy  size.  Most  major  universities 
now  offer  course  degree  programmes  in  O.R.,  based  on  topics  which  were 
unknown  twenty  years  ago,  and  using  a  wide  variety  of  available  textbooks. 

Furthermore,  it  is  increasingly  difficult  to  draw  a  firm  line  between  O.R. 
and  other  disciplines,  as  successful  techniques  are  routinely  taught  and 
used  in  their  fields  of  application,  including  a  variety  of  new  disciplines 
such  as  transportation  and  urban  planning,  waste  management,  energy  analysis 
environmental  engineering,  health  care  systems,  etc.  O.R.  methods  have 
also  had  a  large  influence  on  theoretical  fields,  such  as  mathematics, 
statistics,  and  economics. 

Recent  developments  in  selected  methodological  areas  are  surveyed  to 
indicate  the  variety  and  sophistication  of  O.R.  techniques,  and  current 
research  trends.  Selected  bibliographic  references  provide  an  introduction 
to  the  techniques,  or  to  important  new  developments. 

In  conclusion,  the  current  crises  which  have  appeared  within  the  profession 
are  discussed,  and  the  prospects  of  this  now-mature  field  are  analyzed. 


THE  MALYTIC  METHODS  OF  OPERATIONS  RESEARCH 


by 

William  S.  Jewell 


1.  INTRODUCTION 

When  I  received  the  invitation  to  survey  the  analytical  tools  of 
operational  research  for  this  Meeting,  I  was  extremely  pleased  -  first 
of  all,  for  the  kind  thought  that  I  was  qualified  to  do  so;  and,  secondly, 
for  the  opportunity  to  visit  again  the  country  where  operational  research 
began  over  thirty  years  ago.  However,  as  I  considered  the  explosive 
growth  of  the  field  since  that  time,  I  became  apprehensive  at  the  thought 
of  trying  to  covering  so  diverse  a  topic  in  a  few  words. 

Perhaps  some  of  you  remember  the  actual  situation  in  the  middle  of 
the  50 's  -  the  time  at  which  most  methodology  began  to  be  developed.  Your 
Operational  Research  Society  was  about  ten  years  old,  the  Operations 
Research  Society  of  America  was  about  five,  and  the  Institute  of  Management 
Science  had  just  been  organized.  The  RAND  Corporation  had  just  published 
a  book  entitled,  "A  Million  Random  Digits  with  100,000  Normal  Deviates" 
for  use  in  Monte  Carlo  simulation;  an  electro-mechanical  device  called 
Queuiac  was  promoted  in  the  ORSA  Journal  for  emulating  queueing  problems. 

The  transportation  and  travelling-salesman  problems  had  just  been  described, 
and  neologisms  like  "sub-optimization"  and  "cost-effectiveness  analysis" 
were  appearing.  At  M.I.T.,  where  I  was  pursuing  graduate  studies  in 
engineering,  there  was  no  formal  curriculum  in  operations  research;  our 
only  texts  were  brief  notes  by  P.  M.  Morse,  G.  Kimball,  B.  0.  Koopman, 

G.  P.  Wadsworth  and  others,  RAND  reports,  "The  Theory  of  Games  and 
Economic  Behavior,"  by  J.  von  Neumann  and  0.  Morgenstern,  and  an  esoteric 
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paperback  by  Charnes,  Cooper,  and  Henderson  on  the  optimal  mixing  of 
peanuts  and  cashews.  Our  computer  used  vacuum  tubes,  and  one  walked 
inside  it  for  repairs;  we  solved  what  were  considered  very  large  trans¬ 
portation  problems  (60  plants,  300  customers)  in  about  thirty  minutes. 
Capabilities  improved  with  the  arrival  of  the  first  commercial  computer, 
but  another  student’s  simulation  of  vehicular  tunnel  traffic  took  longer 
than  real  time!  In  1955,  I  attended  my  first  meeting  of  ORSA  at  Columbia 
University.  There  were  two  theoretical  papers  by  R.  E.  Bellman  and 
J.  M.  Danskin,  eight  application  papers  on  production  scheduling  and  urban 
services,  plus  24  contributed  papers  -  I  believe  it  was  the  first  time 
that  two  parallel  sessions  were  required  to  cover  all  the  papers  in  one 
day.  Even  by  1958,  a  comprehensive  bibliography  on  operations  research 
only  contained  3,000  entries  (Case, 1958) . 

In  contrast,  the  next  ORSA/TIMS  meeting  in  Miami  in  November,  1976, 
will,  over  three  days,  have  160  sessions  with  about  980  papers!  There 
are  probably  over  400  texts  in  the  methodologies  of  0.  R.  now  in  print, 
and  a  continually  increasing  number  of  specialty  journals.  A  good  bib¬ 
liography  on  any  of  the  subfields  of  0.  R.  can  easily  include  several 
thousand  entries.  Over  40  colleges  and  universities  in  the  United  States 
now  offer  some  form  of  0.  R.  education;  over  20  of  these  have  named 
departments.  The  technical  capabilities  of  computers  seem  boundless, 
but  our  propensity  to  enlarge  the  boundaries  and  the  scale  of  the  models 
keeps  pace,  as  we  tackle  national,  international,  and  even  global  problems. 

In  every  dimension,  the  field  seems  limitless. 

My  plan  of  attack  to  reduce  the  survey  to  manageable  size  is  as  follows: 
First  of  all,  I  will  describe  briefly  the  major  methodological  areas, 
and  present  what  I  consider  to  be  important  recent  trends,  including 
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references  which  seem  representative  and  interesting,  provide  convenient 
summaries,  or  which  might  serve  as  gateways  for  further  reading.  I  am 
grateful  to  my  many  colleagues  who  have  helped  to  organize  this  biblio¬ 
graphy,  however,  the  final  selection  is  mine,  and  no  claim  to  completeness 
or  showing  historical  priority  is  made. 

Some  methodological  areas,  such  as  control  theory,  are  already  too 
large  to  survey;  other  areas  were  excluded  because  they  seem  to  have  reached 
deadends  (game  theory,  information  theory)  or  because  their  scientific 
basis  is  still  being  developed  (simulation,  management  information  systems, 
urban  and  public  systems).  Except  for  a  brief  section  on  business  models, 
applications  of  methodologies  could,  unfortunately,  not  be  included  in 
this  limited  space. 

Finally,  I  would  like  to  conclude  by  describing  the  influence  of  0.  R. 
methodologies  on  other  disciplines,  considering  some  of  the  crises  facing 
the  profession,  and  giving  my  perspective  on  the  prospects  for  this  now- 
mature  field. 

Operational  research  is,  above  all,  an  optimizing  science,  and  we  begin 
with  a  discussion  of  optimization  methods. 
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2.  UNCONSTRAINED  OPTIMIZATION 

Unconstrained  maximization  (minimization)  is  the  problem  of  finding  a 

:*c  * 

value  X  of  a  vector  x  =  [x-,x„,  x  ]  such  that  f (x  )  >  f (x) 

1  2  n  — 

* 

(or  f (x  )  ^  f(x))  for  all  x  ,  where  f  is  a  given  function,  usually 
analytic.  Most  optimization  methods  require  that  f  have  certain  smooth¬ 
ness  and  shape  properties,  such  as  continuity  and  concavity  (convexity), 

A 

so  that  if  an  x  satisfies  the  necessary  conditions  for  a  local  optimum, 

A 

Vf  =  [3f(x  )/9x^  (i  =  1,2,  n) ]  E  0  plus  2nd-order  conditions,  then 

A 

X  is  also  globally  optimal.  This  enables  one  to  use  local  exploration 
to  lead  to  the  global  optimum. 

Assuming  that  the  gradient  Vf  is  easy  to  calculate,  the  most  popular 

algorithms  are  based  on  the  idea  of  steep  ascent  (descent)  -  that  is,  from 

the  current  solution,  x*^  ,  find  a  new  solution  by  moving  with  (against) 

the  gradient,  x^  =  x*^  +  SVf  ,  where  S  is  a  positive  definite  matrix, 

varying  from  step  to  step,  which  may  have  only  diagonal  terms  (representing 

current  step  size),  or  may  be  more  general,  attempting  to  avoid  the  slow 

* 

convergence  often  encountered  near  x  .  These  methods  are  by  now  quite 
efficient  for  problems  with  several  hundred  variables.  If  derivatives  cannot 
be  easily  computed,  then  a  variety  of  direct  search  methods  are  available. 
Theoretical  details  are  in  (Luenberger,  1973)  and  (Avriel,  1976);  numerical 
comparisons  may  be  found  in  (Himmelblau,  1972);  (Powell,  1970,  1971) 
has  convenient  surveys;  (Wilde,  1964)  details  one-dimensional  search 
procedures . 

If  equality  constraints  are  present,  then  one  can,  in  principle, 
use  Lagrange  multipliers  to  convert  the  problem  to  an  unconstrained  optimum, 
and  apply  gradient  methods  (Kwakernaak  and  Strijbos,  1972),  (Avriel,  1976). 
Alternatively,  one  can  use  nonlinear  programming  methods,  described  below. 
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Unconstrained  optimization  methods  are  useful  primarily  in  the 
simple  pedagogical  models,  or  in  engineering  design  problems.  Their 
main  value  is  as  a  foundation  for  more  complicated  methods  where  equality 
or  inequality  constraints  are  present. 
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3.  LINEAR  PROGRAMMING 

Constrained  linear  optimization  is,  by  any  measure,  one  of  the  most 
successful  "new"  methodologies  of  operational  research.  The  problem  is  to 
optimize  a  linear  function  subject  to  linear  inequality  and  equality 
constraints;  in  the  usual  canonical  form: 


(3.1) 


T 

Max  (Min  c  x) 

X 

Ax  =  b 
X  ^  0  , 


where  x  and  c  are  n-vectors,  b  is  an  m-vector,  and  A  is  an  m  x  n 
matrix  (m  <  n)  .  In  this  form,  linear  inequality  constraints  have  been 
converted  to  equalities  through  the  addition  and  subtraction  of  non¬ 
negative  "slack"  variables,  and  incorporated  into  Ax  =  b  .  Individual 
variable  constraints,  such  as  ,  can  also  be  so  incorporated, 

but  most  computer  codes  have  special  features  to  handle  them  separately, 
so  as  to  keep  down  the  dimension  of  A  . 

The  success  of  linear  programming  is  due  first  of  all  to  its  usefulness 
as  a  model.  The  accounting  world  is  full  of  linearity  assumptions  as  to 

the  costs  of  resources  consumed  and  the  value  of  goods  and  services  produced, 
T 

so  that  c  X  is  a  good  approximation  to  most  management  objectives. 

Many  production  technologies  are  also  linear,  at  least  to  a  first  approxima¬ 
tion,  so  that  portions  of  A  which  represent  conversion  from  activity  j 
to  resource  i  need  only  an  estimate  of  the  conversion  coefficient  a  ; 

ij 

other  portions  of  A  usually  have  large  numbers  of  zero  and  unity  coeffi¬ 
cients  because  of  the  large  number  of  conservation  or  "bookkeeping" 
relations  between  different  activities  -  these  are  intrinsically  linear. 


Finally,  upper  and  lower  bounds  (especially  £  =  0)  on  the  activity 
levels,  X  ,  are  characteristics  of  our  finite  world. 


7 


The  optimal  solution  to  (3.1)  is  determined  almost  completely  by 

* 

the  constraints.  If  there  are  no  ties  for  the  optimal  solution,  x  , 
one  can  show  that  the  optimum  is  determined  by  specifying  a  basic  set  8 
of  m  variables,  x  =  {x^  |  j  e  8}  .  The  values  of  these  variables 
are  determined  by  solving  the  square  system 


(3.2) 


Bx®  =  b 


where  B  is  the  set  of  m  columns  of  A  corresponding  to  8  ;  these  must 

represent  m  linear  independent  vectors  in  m-space  (a  basis)  so  that 

det  B  0  .  These  values  will,  of  course,  be  feasible  only  if  x  ^  0  (and 
B  * 

usually  X  >  0)  .  The  remainder  of  x  is  determined  by  setting  the  n  -  m 

B  I  1 

nonbaslc  variables,  x  =  {x^  |  j  ^  8}  ,  to  zero.  In  geometric  terms, 
selecting  the  n  -  m  nonbasic  variables  determines  a  corner,  or  extreme 
point  of  the  convex  polyhedron  of  feasible  solutions,  {Ax  =  b  ,  x  ^  0}  . 


Since  there  are 


ways  to  select  a  basis  (although  not  all 


B  * 

result  in  feasible  x  ) ,  this  characterization  of  x  suggests  there  are  a 

combinatorial  number  of  basic  feasible  solutions  to  explore. 

:  However,  Dantzig  (1963)  and  his  coworkers  were  able  to  show  that  an 
ascent  (descent)  method,  which  proceeds  from  one  extreme  point  of  the 
constraint  space  to  a  better  extreme  point,  is  a  computationally  efficient 
procedure,  taking  on  the  order  of  2m  -  3m  steps,  rather  than  some 
combinatorial  number  related  to  the  number  of  variables,  n  .  This  fortuitous 
property  of  the  "simplex  method,"  established  through  computational  experience, 
is  the  second  reason  for  its  popularity.  Even  today,  there  is  no  satisfactory 
theory  for  the  rapid  convergence  of  this  method;  theoretical  bounds  on  the 
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number  of  iterations  required  are  extremely  large,  except  for  a  few 
special  problems. 

The  details  of  the  simplex  algorithm  are  quite  straightforward  of 

Interpretation,  although  the  actual  procedures  seem  strange  at  first  glance. 

To  avoid  continued  re- inversion  of  different  m  x  m  matrices,  B  ,B  ,  ...  , 

to  solve  (3.2)  successive  bases  are  chosen  which  differ  only  in  one 

member;  by  using  the  Gauss— Jordan  reduction  method  on  the  full  matrix  A 

(with  b  adjoined),  one  can  easily  check  that  the  current  and  forthcoming 

extremal  solutions  are  feasible.  To  be  assured  that  this  move  increase 
T 

(decreases)  c  x  ,  the  current  solution  is  effectively  substituted  back 
into  the  functional,  and  local  gradients  can  be  read  off  directly  from 
the  new  coefficients  in  front  of  the  nonbasic  variables.  A  "pivot  step" 
(application  of  Gauss-Jordan  reduction  to  one  column  of  data)  then  displays 
the  next  basic  feasible  solution. 

In  economic  terms,  the  choice  of  a  "good"  direction  is  made  by  imputing 

g 

the  unit  profits  c  of  each  current  by  basic  activity  back  to  m  unit 
prices  y  =  [y^,y2,  ...,  y^]  associated  with  each  constraint  by  solving  the 
dual  system  B  y  =  c  ;  the  local  vector  of  gradients  leading  away  from  the 
current  extreme  point  is  then  gotten  from  c  -  A'^y  .  This  leads  to  the 
conceptually  elegant  theory  of  duality,  in  which  it  is  shown  that  (3.1)  is 
equivalent  to  another  linear  program: 

T 

Min  (Max)  b  y 

(3-3)  A^y  ^  (_^)  c 

y  unrestricted, 

in  the  sense  that,  if  one  program  has  a  finite  optimum,  then  so  does  the 
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T  A  T  A 

Other,  and  by  =  c  x  .  The  dual  variables,  y  ,  are  essentially  extended 

Lagrange  multipliers.  Optimality  is  recognized  by  the  "complementary 

A  j  A 

slackness"  condition,  (x  )  (A  y  -  c)  =  0  . 

There  are  many  different  elaborations  upon  the  basic  simplex  method 
that  purport  to  solve  problems  with  special  data  in  more  efficient  ways, 
or  to  fully  exploit  special  structure  or  computational  capabilities,  or 
which  carry  out  post-optimal  sensitivity  analyses.  Full  details  on  this 
by  now  classical  topic  may  be  found  in  almost  any  of  the  approximately  200 
texts  (Gerber,  1974)  on  linear  programming,  such  as  (Dantzig,  1963), 

(Gass,  1958),  (Simonnard,  1966).  FORTRAN  and  ALGOL  programs  are  given  in 
(Kiinzi,  Tzschack,  Zehnder,  1971).  See  also  the  comments  in  (Woolsey, 

1973). 

There  is  a  great  deal  of  difference  between  the  classroom  representa¬ 
tions  of  linear  programs  and  the  actual  computer  codes  (a  term  to  avoid 
confusion  with  "programs")  which  solve  them.  An  l.p.  code  must  not  only 
accept  and  convert  a  variety  of  input  constraints,  it  must  "get  started" 

(by  itself,  or  from  a  prior  solution),  reject  unfeasible  problems,  and 
carry  out  a  variety  of  post-optimal  sensitivity  analyses.  Furthermore,  as 
the  size  of  successful  l.p.  solutions  is  increased,  there  has  been  increasing 
pressure  to  further  increase  the  capabilities  of  advanced  programming  systems. 
This  means  that  a  great  deal  of  attention  must  be  paid  to  what  we  might 
call  the  computer  science  aspects  of  the  program:  allocating  data  between 
different  storage  media,  and  moving  it  about  rapidly;  efficient  methods 
of  storing  inverses  of  sparse  matrices,  updating  them  during  the  pivot  to 
a  new  extreme  point,  and  cleansing  them  of  accumulated  round-off  errors; 

and  finding  the  best  compromise  between  moving  in  the  direction  with  steepest 

T 

gradient  or  the  one  with  greatest  change  in  c  x  .  Details  may  be  found 
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in  (Orchard-Hays,  1968),  (Beale,  1967  and  1970),  (Bonner,  1969);  (Tomlin, 
1972)  and  (White,  1973)  have  more  recent  surveys . 

For  small-scale  linear  optimization,  we  are  clearly  reaching  a  point 
of  diminishing  returns  on  solution  efficiency.  A  problem  with  m  =  200 
which  cost  $1,000  to  solve  in  1956  can  now  be  turned  out  for  under  $20; 
most  of  this  improvement  is  due  simply  to  the  generally  decreasing  cost  of 
digital  computation,  which  trend  will  no  doubt  continue.  However,  the 
demand  for  increased  computational  capabilities  continues  unsatisfied, 
as  programmers  extend  the  size  and  boundaries  of  problem  formulation. 
Currently,  problems  with  m  =  2,000  are  routinely  solved,  and  there  are 
large-scale  systems  which  can  handle  m  >  10,000  ;  since  there  is  no  theory 
to  predict  simplex  method  efficiency,  evaluation  of  new  algorithms  must 
follow  computational  trials.  Occasionally  there  are  surprises;  for 
example,  Harris  (1973)  reports  a  reduction  by  factors  of  2  to  6  in  the 
number  of  iterations  needed  to  solve  problems  with  m  =  2-5000  ,  by 
using  the  concept  of  a  fixed  datum  basis  in  which  to  compute  and  compare 
gradients . 

One  way  of  handling  larger  size  problems  to  take  advantage  of  any 

special  structure  in  the  constraints.  As  mentioned  above,  it  is  trivial 

to  Include  individual  constraints  of  the  forms  £.  <  x.  <  u,  .  In  many 

J  -  J  -  J 

scheduling  and  distribution  problems,  one  encounters  constraints  of  the 
form 


K 


I 

jeJ 


X. 


<  u 
-  K 


K 


where  the  are  nonoverlapping  subsets  of  the  variables.  Problems  with 

a  large  percentage  of  constraints  of  this  type  have  important  savings  in 
computer  time  by  using  the  generalized  upper  bounding  technique  of  Dantzig 
and  Van  Slyke  (1967),  and  recent  l.p.  codes  include  this  capability. 
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Large-scale  multi-time  or  multi-sector  planning  models  have  highly 
structured  matrices  A  ,  with  "block-angular"  and  "staircase"  nonzero  sub¬ 
matrices  (often  with  repetitive  internal  structure) ,  and  zero  elements 
elsewhere.  For  many  years  it  was  thought  that  there  might  be  efficiencies 
by  using  the  simplex  method  separately  on  each  subproblem,  periodically 
reconciling  the  linking  constraints  in  an  overall  simplex  approach;  how¬ 
ever,  in  spite  of  the  large  literature  on  these  methods  (see,  e.g.  (Lasdon, 
1970)),  the  coding  techniques  are  quite  difficult  and  specialized  to  the 
problem  structure,  and  this  approach  is  not  now  an  important  contender  for 
these  problems. 

Another  conceptually  appealing  approach  to  large-scale  l.p.  problems 
which  have  many  separable  subproblems  linked  together  by  a  few  "master" 
constraints  has  been  the  decomposition  method  of  Dantzig  and  Wolfe  (1960). 

In  this  method,  different  values  of  artificial  objective  coefficients  are 
"sent  dotm"  to  the  subproblems,  which  are  solved  individually  (and  hence 
efficiently) ,  producing  a  variety  of  extreme  point  solutions  for  each  subset 
of  variables;  the  "master  program"  is  then  solved  to  satisfy  the  linking 
constraints  by  mixing  these  extreme-point  "plans"  in  an  optimal  way.  The 
dual  solution  to  the  master  program  then  produces  another  set  of  surrogate 
goals  for  the  subprograms,  which  in  tern  provides  other  plans  for  the  master 
optimization.  Perhaps  the  most  important  feature  of  this  approach  is  that  it 
quantifies  the  conditions  under  which  partial  or  completely  decentralized 
economic  planning  can  take  place  (Baumol  and  Fabian,  1964).  However,  as  a 
computational  strategy  for  purely  linear  programs,  decomposition  has  proven 
disappointing.  It  is  still  a  useful  approach  when  the  subproblems  are  not 
linear  programs  (see  the  cutting-stock  problem  below)  or  are  soluble  by 
special  procedures  (such  as  transportation  problems) ,  or  are  linearization 


12 


approaches  to  nonlinear  programming;  Lasdon  (1970)  has  several  applications 
of  the  decomposition  approach. 

In  a  recent  ORSA/TIMS  Panel  (entitled  "Is  Mathematical  Programming 
Moribund?"),  D.  Smith  described  the  four  eras  of  mathematical  programming. 

In  the  '50s,  when  the  simplex  method  was  new  and  incompletely  understood, 
the  basic  problem  was  to  write  l.p.  codes  for  the  different  computers; 
because  of  size  and  speed  limitations,  emphasis  was  also  placed  on  develop¬ 
ment  of  special-purpose  algorithms  for  special  model  structures.  By  the 
'60s,  fast  commercial  computer  codes  were  available  for  production  use,  and 
the  bottleneck  was  in  the  translation  of  output  into  the  management  process; 
feastures  such  as  post-optimal  sensitivity  analysis  were  added,  and  more 
attention  paid  to  simplified  data  entry,  and  to  summary  report  generators. 

As  computational  capabilities  and  management  sophistication  have  increased 
in  the  '70s,  we  find  model  boundaries  expanding,  and  nonlinear,  integer,  and 
decentralized  optimization  capabilities  are  being  added  to  the  computing 
systems.  Finally,  as  we  head  into  the  '80s,  the  routine  solution  of  ex¬ 
tremely  large  problems  raises  enormous  questions  of  data-base  management: 
how  will  data  be  gathered,  stored?  How  will  it  be  checked,  cleansed, 
and  updated?  Who  will  certify  the  results  of  the  optimization  and  what 


methods  will  be  used? 
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4.  NONLINEAR  PROGRAMMING 

Optimization  problems  with  nonlinear  objective  functions  and/or 
nonlinear  inequality  constraints  are  of  increasing  importance  in 
operations  research. 

The  earliest  models  were  of  chemical,  metallurgical,  and  hydraulic/gas 
transmission  processes,  where  the  basic  transformations  are  nonlinear, 

and  in  engineering  design  problems  (Duffin,  Peterson  and  Zener,  1967), 

a . 

(Zener,  1971),  where  empirical  "posynomial"  forms,  TI  ,  are 

encountered.  Quadratic  objectives  arise  naturally  in  least-squares 
approximations  (Golub  and  Saunders,  1970),  in  electrical  networks 
(Dennis,  1959)  and  control  applications  (Luenberger,  1972)  where  energy 
dissipation  is  minimized,  and  in  location  and  space  assignment  problems 
where  Euclidean  distance  is  the  measure  of  optimality.  Even  with  linear 
economic  assumptions,  the  system  objective  may  be  prof it-per-unit- 
(item,  time,  trip,  etc.),  giving  a  linear  fraction,  the  ratio  of  two 
linear  forms,  to  be  optimized.  In  so-called  stochastic  programming, 
the  objective  form  includes  the  expected  cost  of  compensating  for  the 
random  effect  of  a  decision  (Sengupta,  1972),  (Vajda,  1972).  Nonlinear 
constraints  arise  from  similar  considerations. 

In  order  to  retain  the  desirable  property  that  a  local  optimum  is 
also  a  global  optimum,  one  usually  restricts  investigation  to  the  so- 
called  convex  programs,  those  that  can  be  put  in  the  form: 


(4.1) 


Max  (Min)  f (x) 

gj  (x)  ^  0  (j  =  1,2,  .  .  .  ,  p) 

Ax  =  b 


X  >  0 
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(see  Lasdon,  (1970)).  The  more  direct  "approximation  programming" 
approach  replaces  nonlinear  functions  by  their  first-order  Taylor 
series  approximation  expanded  at  the  current  solution,  x^^^  ;  under 
certain  conditions,  the  optimal  solution  to  the  current  l.p.  leads 
to  a  new  estimate  ,  where  another  approximation  is  made,  and 

so  on.  The  (generalized)  "reduced-gradient"  algorithm  partitions  the 
variables  into  basic  and  nonbasic  sets,  as  in  the  simplex  algorithm, 
and  consider  the  optimization  only  in  terms  of  the  latter,  thus 
"projecting  the  gradient."  The  "cutting-plane"  algorithms  turn  a 
problem  into  a  linear  objective  with  nonlinear  constraints,  and  then 
successively  bound  or  support  the  desired  region  by  a  sequence  of 
hyperplanes.  Details  and  further  references  on  these  and  other  simplex- 
like  methods  may  be  found  in  (Beale,  1967),  (Himmelblau,  1972)  and 
(Avriel,  1976).  Most  of  these  algorithms  have  rather  slow  convergence 
since  local  movement  must  be  restricted  to  guarantee  feasibility  or 
convergence.  The  exception  is  the  generalized  reduced  gradient  method 
which  seems  to  remain  among  the  best  nonlinear  codes  (Colville,  1970). 

Turning  now  to  the  methods  which  are  more  closely  related  to 
unconstrained  optimization,  we  note  that,  if  we  have  a  current  solution 
x^*”^  which  is  in  the  interior  of  the  solution  space,  we  can  use  a  steep 
ascent  method,  moving  in  a  sequence  of  straight-line  steps  until  the 
optimum  or  a  constraint  is  reached.  If  the  current  solution  lies  on 
the  boundary  of  a  constraint  (or  if  the  original  formulation  has  linear 
equality  constraints),  then  a  (locally  or  globally)  feasible  direction 
must  be  chosen.  In  the  "gradient  projection"  approach,  due  to  J.  B. 
Rosen,  the  gradient  is  projected  onto  the  active  constraint  hyperplanes. 
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and  a  step  is  taken  in  the  (reduced)  steepest  descent  direction,  until 
the  optimal  point  along  this  line  is  reached,  possibly  with  another 
constraint  becoming  active;  nonlinear  constraints  can  be  handled  by 
approximation,  but  then  extra  steps  to  re-enter  the  feasible  region 
may  be  needed.  On  the  other  hand,  the  "feasible  directions"  method, 
due  to  G.  Zoutendyk,  determines  only  directions  that  are  totally 
feasible,  but  has  difficulty  accommodating  linear  equality  constraints. 

Both  of  these  methods  have  been  considerably  elaborated  and  improved 
by  coupling  them  with  modern  methods  of  steep  ascent.  A  recent  survey 
of  the  many  possible  algorithms  is  in  (Avriel,  1976). 

Penalty  function  methods  operate  differently,  by  incorporating 

the  constraints  into  the  functional  in  various  ways,  and  using  unconstrained 

optimization  techniques.  Exterior  penalty  functions  add  nonlinear  costs 

whenever  the  solution  trajectory  leaves  the  feasible  region.  In  the 

more  popular  Interior  penalty  methods,  nonlinear  "barriers"  are  placed 

in  the  interior  of  the  feasible  region  to  keep  the  optimal  solution 

away  from  the  boundaries;  these  barriers  are  slowly  relaxed  so  that 

#r 

the  sequence  of  unconstrained  optima  converges  to  x  .  For  example, 
if  all  the  constraints  are  of  the  form  g^(x)  ^  0  ,  one  would  maximize 
f(x)  -  1  y  g.^(x)  for  a  decreasing  sequence  of  X  .  This  approach 
has  been  extensively  studied  by  Fiacco  and  McCormick  (1968),  and  seem 
to  be  the  most  successful  way  to  tackle  problems  with  strongly  non¬ 
linear  constraints  (McCormick,  1971).  See  also  (Avriel,  1976)  and  the 
numerical  example  in  (Himmelblau,  1972). 

An  important  special  class  of  nonlinear  programs  are  engineering 
design  problems  in  which  the  objectives  and  constraints  are  composed 
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where  f(x)  is  a  concave  (convex)  function,  and  each  nonlinear 
constraint  is  an  inequality,  formed  only  from  concave  functions. 

This  guarantees  that  the  total  constraint  space  is  a  convex  region, 
and  that  local  search  methods  will  converge,  if  properly  set  up,  to 
the  overall  optimum.  Occasionally,  nonconvex  objectives  of  special 
form  can  be  handled;  otherwise,  one  must  be  content  with  an  x 
which  may  only  be  locally  best . 

A  glance  at  recent  texts  in  nonlinear  programming  (Himmelblau, 

1972),  (Luenberger,  1973),  (Avriel,  1976)  reveals  that  there  is  no 
single  preferred  method,  but  rather  a  variety  of  different  approaches 
suited  to  the  many  different  special  forms  that  (4.1)  can  take.  The 
methods  divide  themselves  naturally  into  those  based  strongly  on  l.p. 
techniques,  those  based  upon  unconstrained  optimization  algorithms, 
and  specially-developed  algorithms. 

In  the  first  category,  we  usually  have  no  (or  few)  nonlinear 

constraints,  and  only  mildly  nonlinear  objectives.  The  first  remark 

T  IT 

is  that  quadratic  programs ,  where  f (x)  =  c  x  +  —  x  Qx  and  Q  is 
a  negative  (positive)  semidefinite  matrix,  can  be  solved  using  variants 
of  the  simplex  method  (Dantzig,  1963),  (Boot,  1964).  Linear  fractional 
programs,  where  f (x)  =  c  x/d  x  is  neither  convex  nor  concave,  can 
be  handled  by  treating  the  denominator  as  a  parametric  variable  (see, 
e.g.  (Lasdon,  1970)).  More  generally,  we  can  use  linear  approximations 
to  the  nonlinear  functions  in  several  different  ways.  The  first  approach 
is  to  use  "grid  linearization,"  with  each  nonlinear  function  recursively 
defined  in  terms  of  local  piece-wise  linear  forms;  the  complete  method 


is  related  to  the  decomposition  algorithm,  and  is  especially  simple  if 

the  nonlinear  function  are  of  separable  type,  f (x)  =  V  f,(x.) 

J  J 
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of  posynomials.  These  problems,  referred  to  as  geometric  programs, 
have  been  extensively  analyzed,  and  special-purpose  algorithms  developed 
(Duffin,  Peterson,  and  Zener,  1967),  (Zener,  1971),  (Beightler  and 
Phillips,  1976).  It  is  also  possible  to  find  approximate  solutions 
using  the  simplex  method  (Ecker  and  Zoracki,  1976). 

Optimality  in  nonlinear  programs  is  recognized  by  constructing 
extended  Lagrangean  multipliers  and  checking  that  the  so-called  "Kuhn- 
Tucker  conditions"  (similar  to  complementary  slackness  in  linear  programs) 
are  satisfied.  These  multipliers  are  essentially  dual  variables  and 
have  similar  economic  Interpretations;  yet,  duality  theory  does  not 
seem  to  play  a  large  role  in  nonlinear  computations.  Part  of  the 
difficulty  is  that  there  are  many  different  possible  dual  formulations 
to  nonlinear  programs,  and,  in  contrast  to  l.p. ,  the  primal  variables, 

X  ,  also  appear  in  the  dual  programs,  thus  preventing  their  Independent 
solution.  The  other  difficulty  is  that  dual  formulations  may  require 
delicate  analysis;  Geoffrlon  (1972)  illustrates  some  of  the  difficulties. 
Convergence  properties  are  also  difficult  to  establish  (Wolfe,  1970), 
(Zangwill,  1969). 

Nonlinear  programming  methods  are  finally  beginning  to  sort  themselves 
out  after  a  period  of  diverse  theoretical  development.  What  is  needed  now 
is  extensive  computational  comparison  on  large-scale  practical  problems 
to  further  match  method  to  problem;  it  is  probably  too  much  to  hope  that 
a  universally  efficient  method  will  ever  be  found. 
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5.  NETWORK  FLOW  MODELS 

If  one  had  to  identify  the  most  popular  application  of  linear 
programming,  it  would  certainly  be  the  network  flow  models,  first  in¬ 
vestigated  systematically  by  Ford  and  Fulkerson  (1962).  In  the 
prototypical  problem,  we  imagine  a  connected  network  made  up  of  nodes 
i—  lj2,  ...,  N  ,  and  A  directed  arcs,  each  labelled  by  an  ordered 
pair  representing  the  start  and  terminal  nodes  for  that  arc.  Thus, 
arc  (i,j)  starts  at  node  i  ,  ends  at  node  j  ,  and  we  suppose  it 

has  flow  X..  ,  limited  by  upper  and  lower  capacities,  1  and  u 
iJ  ij  ij 

and  contributes  a  profit  (cost)  c..x..  to  the  total  operation.  The 

IJ  IJ 

optimization  problem  is  then  to  find  a  maximal  profit  routing  of  flow: 


(5.1) 


Max  (Min)  }  ;  c..x.. 

t  T 


I  ^^ij  ■  ^ji^  "  "^i  . 

1..  <x,.  <u..  . 
ij  -  ij  -  ij 


Here  >  0  [q^  <  0]  represents  external  flow  into  [out  of]  the 

network  at  node  i  ;  the  equality  constraints  represent  "Kirchoff  law" 
conservation  at  each  node  -  summations  are  understood  to  be  only  for 
arcs  actually  connected  there. 

(5.1)  is  clearly  a  linear  program,  but  of  very  special  structure, 

since  the  constraint  matrix  only  contains  O's,  +  I's,  and  -  I's,  and 

I 

exactly  one  of  each  of  the  latter  for  each  variable.  One  can  show 
that  this  implies ; 
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(1)  There  is  exactly  one  redundant  conservation  equation,  and  for 

solvability  ^  =  0  ; 

(2)  The  optimal  solution  is  got  by  adding  and  subtracting  the 

boundary  flows  {q^}  ,  and  is  thus  integral  if  they  and  the 

capacities  {l..,u..}  are  integral; 
ij  iJ 

(3)  The  l.p.  basic  solution  is  related  to  a  network  configuration 
called  a  tree  -  a  subset  of  N  -  1  arcs  which  connects  all  the 
nodes,  and  has  no  loops; 

(4)  "Pivoting"  from  one  extreme  point  solution  to  another  is  related  to 
passing  flow  around  a  loop  in  the  network,  so  the  only  computation  ]abor 
is  to  find  a  "good"  loop  and  stay  feasible. 

Because  of  this  simplicity  in  the  optimal  solution,  it  has  been  possible 
to  develop  fast,  special-purpose  computer  codes  which  can  handle  much 
larger  network  formulations  than  could  be  handled  by  a  general  l.p. 
code  -  10^  nodes  and  10^  arcs  being  handled  routinely  (Glover  and 
Klingman,  1975). 

(5.1)  includes  a  variety  of  very  useful  simpler  models.  For 

example,  if  the  network  consists  only  of  all  possible  links  between 

one  set  of  nodes  with  q^  >  0  (the  plants)  and  another  set  of  nodes 

with  q  <0  (the  customer),  we  have  the  classical  "transportation 
i 

problem";  making  the  two  sets  equal  in  size  and  all  =  +.  1  further 

reduces  (5.1)  to  the  "assignment  problem."  For  a  general  network,  if 

in  (5.1)  c^^  =  1  for  a  certain  arc  (i,j)  ,  zero  for  all  others, 

then  we  have  the  "maximum  (minimum)— flow  problem,"  which  can  be  solved 

by  a  simple  "labelling  method."  (Ford  and  Fulkerson,  1962);  conversely, 

if  all  q.  H  0  ,  but  1..  =  u. .  =1  for  a  certain  arc  (i,j)  ,  then 

ij  ij 
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we  have  the  "longest  (shortest)-route  problem,"  for  which  a  variety 
of  special-purpose  algorithms  are  available  (Dreyfus,  1969).  In  fact, 
most  algorithms  for  the  general  network  problems  are  alternate 
applications  of  max-flow  and  shortest-route  procedures  to  different 
arcs  in  the  network.  Easy  modelling  extensions  include  dynamic  flows, 
capacities  on  nodes,  parametric  studies  (etc.)  (Fulkerson,  1966), 

(Price,  1971). 

The  literature  is  full  of  a  number  of  confusing  references  to 
"primal-dual"  ["Ford-Fulkerson"] ,  "dual,"  "simplex,"  and  "out-of-kilter" 
methods.  These  are,  in  fact,  historical  variants  of  the  same  extreme 
point  method  which  different  only  in  starting  procedures,  selection  of 
new  variables  to  enter  the  basis,  treatment  of  initial  infeasibilities, 
etc.  Nevertheless,  there  seems  to  be  considerable  difference  in  computa¬ 
tional  efficiency  of  the  different  approaches,  and  upon  the  manner  in 
which  labelling  information  is  stored;  see  the  many  studies  of  Glover 
and  Klingman  and  coworkers  (1974a),  (1974b),  (1975).  This  is  important 
because  of  the  many  general  optimization  models  which  have  network-flow 
substructures. 

Incidentally,  linear  network  models  differ  from  general  linear 
programs  in  that  reasonable  bounds  on  the  number  of  iterations  can  be 
obtained  (Edmonds  and  Karp,  1972),  (Dreyfus,  1969).  Zadeh  (1973a), 
(1973b)  gives  some  worst-case  examples. 

The  linear  program  dual  to  (5.1)  has  constraints  of  the  form 
~  Yj  i  (^)  ,  where  the  dual  variables  {y^}  have  an  interesting 

physical  interpretation  as  node  potentials,  obeying  Kirchoff's  potential 
law,  arcs  in  the  optimal  basic  solution. 
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This  dual  program  also  arises  directly  in  the  critical-path  scheduling 
models  (see  below) . 

Because  of  the  success  of  the  initial  special-purpose  algorithms, 

several  extensions  of  flow  models  have  been  proposed,  with  as  much 

of  the  "on  the  network"  simplicity  retained  as  possible.  For  example, 

in  "networks  with  gains,"  we  imagine  that  each  arc  (i,j)  has  a 

multiplier  k. .  which  converts  the  incoming  flow  x, .  to  an  output 
11  iJ 

flow  k..x..  :  this  formulation  includes  a  variety  of  interesting  new 
ij  iJ 

applications  (Jewell,  1962),  but  requires  complicated  labelling  schemes, 
since  the  solutions  are  not  integral,  and  conservation  in-the-large 
is  not  satlfied.  Maurras  (1972)  reports  on  recent  computational  ex¬ 
perience;  Glover  and  Klingman  (1973)  show  that  some  networks  with 
multipliers  can,  in  fact,  be  reduced  by  scaling  to  ordinary  networks, 
and  Truemper  (1976)  discusses  scaling  in  general. 

Another  extension  studied  in  great  detail  is  the  multicommodity 
flow  problem,  in  which  several  types  of  flow  pass  without  mixing  over 
the  network  to  satisfy  their  own  boundary  requirements,  but  are 
mutually  constrained  by  each  arc's  total  flow  capacity.  (Ford  and 
Fulkerson,  1958),  (Jewell,  1966).  Even  with  integral  constraints,  the 
optimal  answer  may  require  fractional  allocations  of  capacity.  Models 
of  this  type  are  important  in  communication  networks  (Frank  and  Frisch, 
1971)  and  road  traffic  problems  (Potts  and  Oliver,  1972).  Recent 
algorithms  are  given  by  Hartman  and  Lasdon  (1972)  and,  with  computational 
experience,  by  Grigoriadis  and  White  (1972). 

It  should  be  mentioned  that  quadratic  profit  (or  loss)  on  arcs 
can  easily  be  handled;  the  procedures  are  a  mixture  of  linear  flow 
procedures  and  the  methods  of  electrical  circuit  theory  (Dennis,  1959). 
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A  variety  of  other  extensions  to  network  flows  have  been  proposed, 
but,  generally,  the  days  of  developing  special-purpose  algorithms  are 
over,  due  to  the  rapidly  increasing  capabilities  of  all-purpose 
mathematical  programming  codes.  Networks  remain,  however,  a  fruitful 
area  of  research  for  other  types  of  models,  such  as  stochastic  flows 
(Frank  and  Frisch,  1971) ,  (Kleinrock,  1976)  and  various  combinatorial 
routing  and  covering  problems  (see  below).  (Bellman,  Cooke,  and 
Lockett,  1970)  give  some  other  clever  network  problems. 

Also  in  the  network  flow  class  of  l.p.g  are  the  critical-path 

scheduling  problems  (Moder  and  Phillips,  1970).  In  this  model,  the 

network  represents  the  precedence  relationships  between  the  different 

jobs  of  a  project.  The  decision  variables  are  dates  {y.l  on  each 

1 

node,  such  that  for  (i,j)  ,  there  is  sufficient  time  to  complete  job 
(i,j)  requiring  time  t..  ;  i.e.  y,  -  y,  >  t.,  .  The  objective  is 
to  minimize  total  elapsed  time  on  the  project.  Since  this  is  exactly 
the  dual  program  to  a  longest-route  problem,  the  solution  procedures 
are  trivial;  however,  the  model  has  found  wide-spread  utility  in  the 
construction  industry. 
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6.  INTEGER  LINEAR  PROGRAMMING 

In  many  optimization  problems  the  assumption  of  a  continuous 

decision  variable  is  untenable  and  one  would  like  an  integral  answer 

for  some  or  all  of  the  variables;  for  instance,  only  an  integer  number 

of  round  trips  can  be  made  by  vehicles,  and  integral  numbers  of 

spare  parts  stocked  for  space  voyages.  Variables  which  can  only  take  on 

values  0  or  1  are  particularly  useful  in  modelling  selection  processes 

with  an  attendant  fixed  cost;  for  example,  if  activity  j  costs  nothing 

if  not  selected,  and  costs  d.  +  c.x,  when  operated  at  level  x.  , 

111  1 

0  <  X.  <  u.  ,  then  we  can  formulate  it  as  part  of  a  ("mixed")  integer 
3  —  3 

program  with  a  cost  d.z,  +  c.x,  ,  and  constraints  0  .S  » 

3  3  3  3  3  3  3 

z.  e  {0,1}  .  The  might  themselves  be  jointly  constrained,  see 

(Ballnski,  1965),  (Garfinkel  and  Nemhauser,  1972a);  other  surveys  of  the 
field  are  (Balinski  and  Spielberg,  1969),  (Greenberg,  1971),  (Garfinkel 
and  Nemhauser,  1972b). 

The  earliest  attempts  to  solve  integer  linear  programs  were  based 

on  the  idea  of  rounding-off  the  variables  in  the  corresponding  continuous 

linear  program;  however,  these  failed  because  it  is  easy  to  construct 

examples  where  the  optimal  Integer  answer  is  not  the  feasible  integer 

point  nearest  to  the  best  l.p.  solution  -  in  fact,  can  be  arbitrarily 

far  away.  Only  after  the  initial  paper  of  Gomory  (1958)  were  exact 

solutions  possible.  His  idea  was  that  of  adding  additional  constraints 

to  sequentially  generate  the  convex  hull  of  feasible  integer  points; 

these  "cuts"  remove  part  of  the  original  solution  space  but  do  not 

remove  any  integer  solutions.  After  again  optimizing  using  the  simplex 

* 

method,  the  new  non- integral  x  suggests  other  cuts,  and  so  forth. 

Many  different  methods  of  generating  these  cutting  planes  are  now  available. 
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but,  generally  these  methods  have  slow  convergence,  and  have  been  tested 
mostly  on  problems  of  less  than  a  hundred  integer  variables. 

The  most  efficient  methods  for  general  integer  programs  are 
currently  based  upon  implicit  enumeration  techniques  -  the  so-called 
"branch-and-bound"  method  (Agin,  1966),  (Lawler  and  Wood,  1966), 

(Mitten,  1970)  or  "progressive— separation— and  evaluation"  procedure 
(Bertier  and  Roy,  1964),  (Roy,  Benayoun  and  Tergny,  1970).  The  basic 
idea  can  be  illustrated  by  considering  the  integer  program! 

T 

Max  P  =  c  X 

(6.1)  Ax  <=  b 

x^  e  {0,1,2,  ...,  uj  (j  =  1,2,  ...,  n)  . 

An  upper  bound  on  P  =  c  x  can  obviously  be  obtained  by  solving 
the  corresponding  l.p.  with  the  integer  constraints  replaced  by 

^ j  .1  ^ j  {j  “  lj2,  ...,  n)  .  Now,  pick  a  certain  variable  to  "arbitrate" 
or  branch  upon"  -  say  x^  .  Problem  (6.1)  is  "separated"  into  u^  +  1 
distinct  integer  programs  in  which  x^^  is  fixed  at  its  possible  values 
0,1,2,  ...,  Uj^  ;  the  objective  function  for  each  of  these  subproblems 
can  then  be  bounded  from  above  by  a  linear  program  in  which  0  <  x  <  u 

-  j  -  j 

(j  “  2,3,  ...,  n)  -  and  these  solutions  are  usually  within  a  few  simplex 
steps  of  each  other,  as  x^  is  varied  parametrically  through  its  integer 
values.  To  proceed,  the  "best"  choice  of  x^  ,  in  terms  of  the  various 
bounds  on  P  ,  is  taken,  and  a  different  variable  is  chosen  for  further 
exploration,  generating  a  new  sequence  of  subproblems  in  which  two  variables 
are  now  fixed  at  integer  values.  At  successive  steps,  one  takes  the 
"best"  overall  candidate  solution,  irrespective  of  the  number  of  arbitrated 
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variables.  Under  very  general  conditions,  one  can  show  that  this  procedure 
terminates  when  the  first  solution  with  all  variables  fixed  is  obtained. 
This  enumerative  approach  could,  in  principle,  require  exploration  of 
all  possible  II  (u^  +  1)  solutions,  but  in  practice  behaves  rather  well 
if  good  rules  for  the  selection  of  successive  variables  are  used.  A 
convenient  framework  for  explaining  the  various  approaches  is  in 
(Geoffrion  and  Marsten,  1972).  Commercial  mixed-integer  codes  are  now 
undergoing  extensive  computational  testing,  with  encouraging  results 
for  problems  with  several  hundred  integer  variables  and  several  thousand 
constraints  (Roy,  Benayoun  and  Tergny,  1970),  (Benichon,  Gauthier, 

Girodet,  Hentges ,  Ribiere  and  Vincent,  1971),  (Mitra,  1973),  (Forrest, 

Hirst  and  Tomlin,  1974).  It  is  interesting  that  the  most  successful 
solutions  use  a  number  of  heuristic  procedures,  and  depend  strongly  on 
the  way  in  which  the  problem  is  formulated  (Geoffrion,  1976). 

Naturally  there  are  speedier  special-purpose  algorithms  available 
for  specific  models,  for  Instance,  if  the  l.p.s  used  to  determine 
the  bounds  are  of  the  network  flow  type.  (Balinski  and  Spielberg,  1969). 
Geoffrion  and  Graves  (1974)  report  a  successful  application  of  an  older 
method  due  to  Benders  (1962)  to  a  very  large  warehouse-location  and 
multicommodity  distribution  problem.  Other  special-purpose  methods 
and  models  are  described  below. 

Finally,  Kalvaitis  and  Posgay  (1974)  describe  a  very  successful 
commercial  application  of  Integer  programming,  while  Woolsey  (1972) 
injects  a  cautionary  note. 
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7.  COMBINATORIAL  OPTIMIZATION 

The  boundary  between  integer  programming  and  combinatorics  is 
not  a  precise  one,  since  many  of  the  problems  we  consider  here  have 
formulations  like  (6.1),  with  0  or  +  1  constraint  coefficients. 

However,  combinatoric  problems  are  generally  more  "puzzle-like," 
are  either  very  simple  or  very  difficult,  and  usually  require  special 
algorithmic  development .  The  most  important  ones  are  related  to 
network  applications. 

To  give  some  examples,  imagine  a  network  (or,  if  you  prefer,  a 
connected  graph)  with  A  undirected  arcs  and  N  nodes.  Each  arc 
has  a  positive  unit  cost;  the  total  cost  of  "using"  a  certain  subset 
of  arcs  is  the  sum  of  the  unit  costs  associated  with  those  arcs. 

Now  consider  the  following  different  problems; 

(1)  Given  two  specific  nodes,  find  the  least-cost  (shortest)  path 
(sequence  of  arcs  with  nodes  in  common)  from  one  node  to  the 
other ; 

(2)  Repeat  (1),  but  find  the  2nd, 3rd,  ...,  kth  shortest  paths; 

(3)  Find  the  least  cost  subset  of  arcs  which  will  connect  all  nodes 
to  each  other; 

(4)  Find  the  minimal-cost  tour  which  passes  through  all  nodes  at 
least  once,  returning  to  the  starting  node; 

(5)  Find  the  minimal-cost  tour  which  traverses  each  arc  at  least  once; 

The  shortest-path  problem  (trivially  extended  to  undirected  arcs) 

has  already  been  discussed  as  a  special  linear  program  of  the  network 

2 

flow  type;  it  can  be  solved  in  the  order  of  N  steps  using  a  special 
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dynamic-programming-type  algorithm;  problem  (2)  also  has  an  efficient 
dynamic-programming  formulation.  (Dreyfus,  1969). 

In  problem  (3) ,  one  can  easily  show  that  the  desired  configuration 
is  a  (spanning)  tree,  N  -  1  arcs  which  connect  all  nodes,  and  has  no 
loops.  This  problem  cannot  be  posed  as  an  l.p.,  and  yet  is  susceptible  to 
almost  any  kind  of  "greedy"  heuristic:  for  example,  pick  the  least-cost 
arc,  then  add  to  it  the  next-lowest-cost  arc  not  forming  a  loop,  then 
add  to  those  the  next-lowest- cost-arc  not  forming  a  loop,  ...  and  so  on, 
until  a  tree  is  obtained  in  at  most  A  steps.  This  model  is  useful 
in  a  variety  of  communication  problems  (Frank  and  Frisch,  1971),  (Pierce, 
1974). 

Problem  number  (4)  is  the  well-known  "travelling  salesman"  problem, 

and  is  fundamentally  more  difficult  than  (3).  Certain  general  theorems 

are  known  (Bellmore  and  Nemhauser,  1968);  for  example,  if  the  unit  costs 

obey  a  "triangle  inequality"  (it  is  always  cheaper  to  go  from  one  city 

to  another  in  one  step  than  in  two),  then  the  optimal  tour  is  a  "Hamiltonian 

cycle"  -  a  circuit  of  N  arcs  visiting  each  city  once  and  only  once. 

A  variety  of  different  approaches  have  been  proposed  for  this  problem: 

dynamic  programming,  where  storage  bottlenecks  limit  the  size  of  the  problem; 

integer  linear  programming  formulations,  which  required  the  addition  of 
N-1 

2  -  1  constraints  -  later  improved  by  cutting-plane  algorithms;  and 

branch-and-bound  algorithms  which  have  variable  performance  depending  on 
the  heuristics  chosen.  Bellmore  and  Nemhauser  (1968)  provide  a  summary; 
branch-and-bounding  appears  already  as  the  best  method,  but  only 
problems  with  N  <  100  were  solved  exactly.  Held  and  Karp  (1970) , 
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(1971)  have  found  that  sharper  bounds,  derived  from  solving  a  related 
spanning  tree  problem,  can  produce  Important  efficiencies.  Their 
approach  has  been  further  improved  by  Hansen  and  Krarup,  (1974),  and, 

for  directed  arc  networks,  by  Smith,  Srinivasan  and  Thompson  (1975); 

3.5 

computation  times  seem  to  vary  about  as  N  for  small  problems, 
but  N  =  200  is  about  the  limit  for  exact  solutions.  (Webb,  1971) 
and  (Lim  and  Kernighan,  1973)  show  how  to  obtain  good  approximate 
solutions  for  larger  problems. 

Problem  number  (5)  is  called  the  "Chinese  postman"  problem.  In 
the  undirected  arc  network  where  every  node  has  an  even  number  of  arcs, 
there  exists  an  "Euler  tour,"  a  tour  which  passes  through  each  arc 
only  once;  this  is  then  optimal.  In  the  contrary  case,  extra  trips 
are  necessary  to  pass  through  nodes  of  odd  degree;  this  is  done 
through  an  associated  integer  "matching"  problem  for  which  good 
computational  experience  is  available  (Edmonds  and  Johnson,  1973). 

Both  the  travelling-salesman  and  Chinese-postman  problems  are 
important  as  building  blocks  in  realistic  routing  applications. 

Orloff  (1974a)  has  synthesized  these  methods  to  solve  a  general 
routing  problem  (in  which  the  minimum-cost  tour  is  to  visit  a  subset 
of  the  nodes  and  cover  a  subset  of  the  arcs) ;  this  has  important 
application  to  the  problem  of  routing  a  fleet  of  vehicles  out  of  a 
central  facility,  as  in  school-but-routing,  and  refuse  scavenging 
(Orloff,  1974b).  See  also  (Bennett  and  Gazis,  1972),  (Beltrami  and 
Bodin,  1974)  and  the  survey  in  (Bodin,  1975). 

The  concept  of  choosing  an  optimal  tour  to  pass  through  certain 
nodes  or  arcs  of  a  network  can  be  generalized  to  the  combinatorial 
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problems  of  "set  covering,"  in  which  costly  subsets  are  to  be  chosen 
(from  a  given  family  of  subsets)  so  as  to  span  the  original  set  of 
elements  at  minimal  total  cost;  if  the  selected  subsets  are  also  to 
be  disjoint,  it  becomes  a  problem  of  "set  partitioning."  See 
(Garfinkel  and  Nemhauser,  1972b,  1972c)  and  (Balas  and  Padberg,  1976) 
for  surveys  of  methods  and  references;  (Marsten,  1974)  reports  recent 
computational  experience. 

These  set  covering/partitioning  models  can  be  applied  to  a  variety 
of  discrete  selection  problems.  Perhaps  their  most  useful  application 
to  date  has  been  to  the  problem  of  scheduling  airline  crews  to  "cover" 
a  flight  schedule  at  minimal  salary,  living-expense,  and  "deadheading" 
costs,  subject  to  various  restrictions  on  work-and-rest  times,  company 
and  union  requirements,  etc.  The  total  problem  is  quite  complex,  and 
a  variety  of  approaches  have  been  proposed  at  intra-industry  meeetings; 
a  convenient  summary  is  in  (Arabeyre,  Fearnley,  Sterger  and  leather,  1969). 

Another  highly  visible  use  of  integer  programming  is  in  the  problem 
of  determining  political  districts  so  as  to  achieve  equity  in  terms  of 
absolute  deviations  of  district  population  from  an  overall  mean.  Minimiz¬ 
ing  the  sum  of  such  deviations  over  all  districts  is  a  set  partitioning 
problem;  if  the  objective  is  to  minimize  the  largest  of  such  deviations, 
we  have  a  "bottleneck  problem"  for  which  a  branch- and-bound  procedure 
has  been  tested  on  a  state  distincting  problem  with  40  indivisible  popula¬ 
tion  units.  (Garfinkel  and  Nemhauser,  1970).  A  related  political 
topic  is  the  problem  of  determining  a  fair  apportionment  of  representa¬ 
tives  between  political  units;  Balinski  and  Young  has  developed 
a  new  "quota  method"  using  Integer  programming  ideas,  and  have  applied 
it  to  the  U.S.  Congress  (1975)  and  the  European  Parliment  (1976). 
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Finally,  no  discussion  of  combinatorics  would  be  complete  without 
a  discussion  of  new  results  on  computational  complexity.  There  are 
certain  problems,  such  as  maximal-flow,  shortest-route ,  assignment,  and 
minimal  spanning-tree  problems,  where  one  can  guarantee  that  the  number 
of  solution  steps  is  less  than  some  polynomial  function  of  the  problem 
parameters;  i.e.,  a  "polynomial-time-bounded  algorithm"  exists 
(Edmonds,  1965).  Using  modern  concepts  of  algorithmic  analysis  (see, 
e.g.  (Aho,  Hopcraft  and  Ullman,  1975)),  Karp  has  shown  that  a  variety 
of  other  combinatorial  problems  are  equivalent  in  the  sense  that  if 
any  one  of  them  can  be  solved,  then  there  is  a  polynomial-time-bounded 
transformation  which  will  solve  any  of  the  others;  it  follows  then  that 
all  or  none  of  the  members  of  this  "NP  complete  class"  are  solvable 
in  polynomial  time  (Karp,  1972),  (Karp,  1975).  This  class  is  quite 
wide  and  contains  the  travelling-salesman,  integer-linear  programming, 
knapsack  problems,  and  set  covering  and  partitioning  problems;  since 
all  of  these  are  computationally  difficult,  we  suspect  that  none  of 
this  is  polynomial-time-bounded.  Of  course,  this  does  not  mean  that 
there  cannot  be  efficient  algorithms  for  moderate-size  problems,  or 
even  that  an  "average"  problem  cannot  be  solved  during  a  time  which  is 
a  polynomial  function  of  its  size,  as  we  have  seen.  In  fact,  many  heuristic 
algorithms  have  already  been  surprisingly  successful  in  solving  actual 
combinatorial  problems.  An  exciting  new  line  of  research  is  now  trying 
to  quantify  this  success  by  looking  at  the  proximate  success  of  heuristics 
on  distributions  of  problem  parameters;  in  many  cases  one  can  guarantee 
that  all  but  small  percentage  of  such  problems  will  be  optimally  solved 
by  a  fast  algorithm  (Karp,  1976). 
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8 .  DYNAMIC  PROGRAMMING 


Dynamic  programming  is  not  so  much  a  method  of  optimization  as  it  is 

a  framework  in  which  to  efficiently  analyze  loosely-coupled,  repetitive 

decision  problems.  Typically,  these  problems  arise  in  dynamic  models  (or 

serial  processes)  where  the  decisions  made  at  one  Instant  (or  stage)  give 

rise  to  a  similar  problem  at  a  later  time  (the  next  stage)  ,  in  which  a 

parameter  or  the  state  of  the  system  has  changed. 

A  typical  model  is  the  resource  allocation  problem,  in  which  activity 

j  ,  operating  at  level  x,  restricted  to  some  set  of  values  S,  ,  uses  up 

1  J 

an  amount  a^  >  0  of  resource,  and  generates  revenue  r^(x^)  , 

(j  =  1,2,  ...,  N)  .  Assuming  there  are  B  total  units  of  resource,  the 
global  optimization  problem  is: 


N 


Max  'l  r.(x,) 


j=l 


1  1 


(8.1) 


y  a.x.  <  B 

Li  -I  -I  - 


3=1 


3  3 


X j  £  S j  (j  1,2,  ...,  N)  . 


If  the  return  functions  are  linear,  and  is  an  interval  on  the  real 

line,  this  is  a  simple  l.p. ;  more  general  problems  require  special  handling, 
even  with  just  one  constraint.  In  the  dynamic  programming  approach,  we 
solve  the  problem  in  stages,  usually  beginning  with  the  last.  Let 
be  the  optimal  return  from  the  Nth  activity,  assuming  b  units  of  resource 
are  made  available  to  it;  this  is  got  from  the  simple  optimization: 
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(8.2) 


f^(b)  =  Max  r^(xj^) 


Vn  -  ^ 


which  is  solved  parametrically,  for  all  values  of  0  <_  b  _<  B  .  Knowing 

“k 

f^(b)  (and  having  recorded  the  optimal  decision  x^(b))  ,  we  now  proceed 
to  the  determination  of  optimal  return  from  the  last  two 

activities,  assuming  b  units  of  resource  are  available  for  hath.  Since 
activity  N  -  1  uses  up  of  this  resource,  we  have  again  a  one- 

dimensional  optimization: 


(8.3) 


f^_l(b)  =  Max  +  fjj(b  -  a^j_^x^_3_) 


^N-l^N-1  -  ^ 


^N-1  ^  ^N-1 


which  is  solved  parametrically  for  0  _<  b  ^  B  .  This  process  is  carried 
out  successively  for  all  preceding  stages  until  solving  f^(b)  for  b  =  B 
gives  the  optimal  total  return.  Note  that  the  optimization  difficulty  is 
reduced  to  that  of  a  one-dimensional  problem,  but  that,  in  exchange,  a 
sequence  of  optimal  returns  (and  policies)  must  be  stored  for  all  values  of 
b  .  Thus,  large  dynamic  programs  are  typically  storage-limited,  and  it  is 
difficult  to  adequately  handle  problems  with  more  than  2  or  3  linking 
constraints.  In  other  formulations,  the  result  of  one  stage's  optimization 
is  to  leave  the  system  in  a  different  abstract  state  (such  as  location  and 
position  in  space),  rather  than  with  a  diminished  scalar  variable;  here  the 
"curse  of  dimensionality"  requires  one  to  quantize  the  state  space  rather 
grossly  to  get  an  initial  approximation  in  reasonable  computation  storage. 
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and  then  to  successively  refine  the  space  (Larson,  1968).  Bertele  and 
Brioschi  (1972)  have  analyzed  nonserial  models  which  require  a  different 
kind  of  successive  approximation.  Dirickx  and  Jennergren  (1975)  examine 
myopic  policies. 

The  observation  that  many  dynamic  models  could  be  reduced  to  a  series 
of  one-stage  parametric  problems  is  due  to  R.  Bellman,  who  called  it  an 
optimality  principle:  "An  optimal  policy  has  the  property  that,  whatever 
the  initial  state  and  initial  decision  are,  the  remaining  decision  must 
constitute  an  optimal  policy  with  regard  to  the  state  resulting  from  the 
first  decision."  (Bellman,  1957).  A  variety  of  different  formulations 
which  use  this  principle  can  be  found  in  the  still-useful  texts  (Bellman, 
1957)  and  (Bellman  and  Dreyfus,  1962). 

Despite  their  inherent  limitations,  dynamic  programs  are  extremely 
useful  for  problems  of  moderate  size,  or  as  subroutines  in  larger  problems. 
The  premier  example  of  this  is  the  cutting  stock  or  trim  problem,  thoroughly 
investigated  by  Gilmore  and  Gomory  (1961),  (1963),  (1965),  (1966).  Imagine 
that  an  order  for  different  numbers  of  different  lengths  is  to  be  filled  by 
cutting  from  a  number  of  larger,  standard  lengths;  the  problem  is  to  mini¬ 
mize  wastage  of  stock.  If  the  size  of  the  order  is  large,  an  approximate 
solution  can  be  found  by  using  a  linear  program  in  which  the  variables  are 
all  the  different  possible  patterns  of  cutting  the  standard  lengths.  Instead 
of  enumerating  (the  combinatorial  number  of)  all  such  patterns,  Gilmore  and 
Gomory  generate  new  candidate  patterns  by  solving  a  related  "knapsack 
problem,"  which  is  the  name  given  to  (8.1)  when  the  returns  are  linear, 

r  (x  )  =  r.  •  X.  ,  and  the  variables  are  Integer,  S  =  {0,1,2,  ...,  u . }  . 
Jill  J  ^ 

In  this  way,  they  alternate  between  linear  and  dynamic  programs  to  solve  a 
complex  problem  with  many  industrial  applications. 


The  knapsack  problem  is  an  important  model  in  its  own  right,  both 
because  of  its  usefulness  as  a  building-block  in  realistic  packing  situa¬ 


tions,  and  because  of  its  deceptive  simplicity.  For  example,  if  the 
activities  (items  to  be  selected  for  the  knapsack)  are  arranged  in  de¬ 
creasing  order  ^  ^  '■*  (decreasing  value  per  unit  resource 

(space  or  weight)  consumed),  and  if  can  be  fractional,  0  ^  ^  > 

then  the  solution  is  trivial:  take  x^  =  Min  (u^,B/a^)  ;  = 

Min  (u2,(B  -  a^x^)/a2)  ;  etc.  It  might  be  thought  that  is  "greedy" 
algorithm  might  also  extend  to  the  integer  case  by  making  the  obvious 
modification  in  the  2nd  term;  unfortunately,  this  is  only  true  for  very 
special  data  sets  (Magazine,  Nemhauser  and  Trotter,  1973),  and  we  know  that 
the  general  knapsack  problem  belongs  to  the  potentially  difficult  NP- 
complete  class  of  problems.  Many  other  integer  programming  methods,  such 
as  cutting-plane  and  branch-and-bound  algorithms,  have  been  proposed 
(Garfinkel  and  Nemhauser,  1972a). 

Dynamic  programming  is  also  useful  as  a  theoretical  tool,  in  proving 
the  optimality  of  certain  forms  of  decision,  rather  than  assuming  the  form, 
and  merely  setting  the  control  parameters  optimally.  Perhaps,  the  most 
important  result  of  this  kind  was  the  proof  that  two-bin  Inventory  control 
policy  was  optimal  under  certain  cost  and  demand  assumptions  (Arrow,  Karlin 
and  Scarf,  1958).  Also,  one  can  often  show  that  dynamic  optimization 
problems  in  operations  research  have  a  limiting  behavior,  in  the  sense  that, 
as  the  planning  horizon  increases  without  limit,  the  optimal  total  return 
may  be  bounded  in  value  (if  returns  discounted  over  time)  or  be  bounded  by 
a  linear  function  of  time  (if  undiscounted) ,  and  the  optimal  decision  may 
be  stationary;  the  optimality  principle  then  becomes  a  recursion  relation¬ 


ship  which  can  be  solved  Iteratively.  A  well-studied  example  of  this  is 
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the  model  of  Markov  programming,  in  which  a  decision  take  a  process  in 

state  i  at  time  t  into  state  j  at  time  t  +  1  with  probability 

p,.(z  )  ,  earning  a  reward  r..(z  )  ,  (i,j  =  1,2,  . . . ,  N)  (t  =  0,1,2,  ...) 
ij  t  t 

thus,  if  the  policy  is  fixed  for  all  t  ,  the  process  follows  a  Markov  chain 
and  the  mean  total  reward  is  asymptotically  proportional  to  t  .  Many 
interesting  decision  problems  can  be  formulated  in  this  framework  (Howard, 
1960)  and  there  are  numerous  extensions  (Jewell,  1963),  (White,  1969), 

(Mine  and  Osaki,  1970).  If  z^  is  a  discrete  control  action  and  N  is 
finite,  this  problem  can  be  solved  a  number  of  different  ways,  including 
linear  programming.  Problems  with  continuous  decision  and  state  spaces 
requires  a  certain  amount  of  delicate  analysis,  and  here  we  must  say  that 
the  theory  has  far  outstripped  the  applications  (Porteus,  1975). 

Interesting  applications  of  dynamic  programming  are  appearing  regularly 
in  the  literature,  particularly  in  various  investment  and  consumption 
(Hakansson,  1970),  allocation  (Herman,  Lieberman  and  Ross,  1975),  and 
"stopping”  (Leonardz ,  1973)  models. 
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9.  CONTROL  THEORY 

There  is  an  intimate  relation  between  the  theory  of  dynamic  programming 
and  recent  developments  in  control  theory,  which  may  loosely  be  described  as 
optimization  of  a  system  of  differential  (or  difference)  equations.  A  sur¬ 
vey  of  this  area  would  take  many  more  pages ,  and  we  content  ourselves  with 
references  to  (Bellman,  1967,  1971)  for  a  dynamic  programming  presentation, 
and  to  (Canon,  Cullum,  and  Polak,  1970)  and  (Luenberger,  1972)  for  discussion 
of  the  interaction  with  mathematical  programming. 
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10.  MULTIPLE  OBJECTIVES  AND  DECISION-MAKING  UNDER  UNCERTAINTY 

Analysts  have  long  realized  that  optimization  of  a  single  fixed  objec¬ 
tive  is  not  responsive  to  the  needs  of  decision-making  in  the  real  world, 
where  many  different  conditions  influence  choice.  It  is  of  course  possible 
to  explore  several  different  objectives  simultaneously  to  evaluate  trade-offs; 
this  is  especially  easy  in  linear  programming  with  only  a  few  conflicting  ob¬ 
jectives  (Zeleny,  1974) .  However,  the  general  problem  is  quite  difficult, 
since  the  decision-maker  may  not  be  able  to  define  his  preference  space  pre¬ 
cisely  until  faced  with  actual  comparisons.  A  variety  of  different  models 
may  be  found  in  (Cochrane  and  Zeleny,  1973);  (Roy,  1971)  is  a  synthesis  of  the 
different  methodologies. 

Somewhat  the  same  problem  arises  in  decision-making  under  uncertainty 
(Hertz,  1973) .  In  some  instances,  one  can  justify  a  single  objective,  such 
as  maximizing  a  mean  value  or  the  probability  of  gaining  a  fixed  sum.  Usually, 
however,  several  aspects  of  the  distribution  of  outcome  seem  important;  for 
example,  in  the  E-V  approach  of  Markowitz  (1959),  one  examines  the  trade-off 
between  mean  and  variance.  More  generally,  the  problem  is  one  of  comparing 
two  distributions.  Some  results  can  be  obtained  through  ideas  of  stochastic 
dominance,  but  the  preferred  approach  seems  to  be  through  utility  theory,  as 
developed  by  Von  Neumann  and  Morgenstern.  Given  a  choice  between  several 
distributions  of  random  outcomes  {p^(x)}  ,  their  result  states  that,  given 
three  reasonable  hypotheses  which  a  Rational  Economic  Man  might  follow  when 
constructing  preferences  among  these  gambles,  the  preferences  can  always  be 
represented  in  terms  of  a  nondecreasing  utility  function,  u(x)  ,  idiosyn- 
chratic  to  the  decision-maker,  by  ranking  the  distributions  according  to  the 
expected  utility  of  the  i  gamble,  ~  f u(x)p^(x)dx  .  This  approach  is 
well-explained  in  (Borch, 1968)  (White,  1969)  (Raiffa,  1970) .  Although  there 
have  been  numerous  objections  to  utility  theory  (for  example,  an  E-V  decision- 
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maker  does  not  satisfy  the  hypotheses  of  the  theory),  it  seems  very  difficult 
to  modify.  And,  it  does  satisfactorily  explain  certain  observed  behavior, 
such  as  paying  a  premium  over  the  mean  loss  (profit)  for  insurance  (a  lot¬ 
tery  ticket)  if  one  is  risk- averse  (risk-seeking).  There  has  been  a  great 
deal  of  attention  to  the  problem  of  defining  multi-attribute  preferences, 
and  the  construction  of  an  overall  utility  function;  (Keeney  and  Raiffa,  1976) 
is  the  definitive  text. 

Utility  theory  finds  usage  in  the  models  of  "decision  analysis",  a  new 
guise  for  statistical  decision  theory  which  emphasizes  the  formal  process  of 
laying  out  a  decision  tree,  the  estimation  of  the  probabilities  associated 
with  nature  s  plays,  the  estimation  of  the  utility  of  the  terminal  outcomes, 
and  the  use  of  Bayes'  law  and  dynamic  programming  to  calculate  optimal  stra¬ 
tegies.  (Raiffa,  1970)  Decision  analysis  is  particularly  useful  as  a  pedagog¬ 
ical  framework,  and  as  means  of  structuring  communication  between  the  analyst 
and  the  decision-maker.  Suggestive  applications  appear  in  (Grayson,  1960) 
(drilling  decisions),  (de  Neufville  and  Kenney,  1972)  (airport  development), 
and  (Hax  and  Wlig,  1976)  (capital  investment). 
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11.  STOCHASTIC  PROCESSES  AND  MODELS 

Much  of  the  early  literature  in  OR  was  devoted  to  the  study  of  random 
processes,  both  because  of  rapid  developments  in  the  50' s,  in  communications 
theory,  but  also  because  it  was  not  usual  to  be  trained  in  this  area.  Now 
the  situation  is  reversed;  we  take  for  granted  that  the  OR  specialist  has  had 
at  least  two  courses  in  stochastic  processes,  and  there  seem  to  be  few  useful 
new  theoretical  developments.  (^inlar,  1975b)  is  an  example  of  a  modern  theo¬ 
retical  text. 

The  most  ubiquitous  model  is,  of  course,  the  Markov  process,  especially 
in  its  discrete- time,  discrete  state-space  version,  the  Markov  chain.  In 
about  every  field  of  application,  one  can  find  a  Markov  chain,  possibly  im¬ 
bedded  in  a  more  complicated  process  (Kendall,  1953),  used  to  describe  suc¬ 
cessive  transitions  between  states.  In  a  certain  sense,  it  represents  the 
first  order  of  dependence  up  from  a  purely  independent- transition  process, 
and  its  modelling  success  is  due  to  the  fact  that  higher  order  dependencies 
are  rarely  needed.  The  properties  of  Markov  chains  have  been  well-understood 
for  twenty  years,  thanks  to  the  still  excellent  book  by  Feller  (1967). 

The  other  useful  model  is  the  renewal  process,  which  describes  point 
processes  as  generated  by  independent,  identically  distributed  random  in¬ 
tervals;  this  is  an  obvious  model  in  reliability,  where  failed  items  are 
immediately  replaced  by  new,  similar  items,  but  is  also  useful  as  a  model 
for  other  processes,  such  as  arrival  of  customers  at  a  queue.  (Cox,  1962) 

is  a  good  introduction  to  the  field;  the  full  generality  of  renewal  argu¬ 

ments  and  the  various  limit  theorems  are  covered  in  (Feller,  1971). 

By  combining  Markov  chains  and  renewal  theory,  (so  that  a  transition 
between  two  states  i  and  j  takes  a  random  duration  sampled  from  a  dis¬ 
tribution  depending  on  i  and  j)  ,  we  obtain  the  very  useful  Markov-re- 

newal  (or  semi-Markov)  processes.  The  theory  is  only  mildly  more  complicated. 
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and  subsumes  many  early  elaborations  on  the  basic  processes.  (^inlar, 

1975a)  surveys  the  field;  (Teugels,  1976)  is  a  bibliography. 

The  other  important  extension  to  these  basic  theories  is  the  addition 
of  economic  functions,  called  rewards  or  potentials,  for  use  in  optimization. 
For  example,  if  a  transition  between  states  i  and  j  took  t  units  of 
time,  we  might  generate  a  profit  (t)  at  the  end  of  the  interval,  and 

add  it  to  other  rewards  earned  from  previous  transitions.  The  mathematical 
details  are  easy,  and  are  already  being  included  in  introductory  texts  (Ross, 
1970). 

Other  stochastic  topics,  such  as  random  walks,  branching  processes  and 
diffusion  processes  find  special  uses,  particularly  in  queueing  theory 
(Gaver,  1968)  (Newell,  1971) ,  and  in  attempts  to  model  stock  market  behavior 
(Fama,  1970),  but  few  other  applications. 
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12.  QUEUEING  THEORY 

The  study  of  congestion  in  service  systems  was  very  popular  in 
the  '50s  and  '60s.  Although  the  basic  modelling  had  been  carried  out 
many  years  previously  by  A.  K.  Erlang,  and  others,  for  problems  of 
telephone  traffic,  the  subsequent  development  of  queueing  theory 
showed  the  essential  similarity  between  congestion  and  waiting-time 
phenomena  in  such  diverse  applications  as  road  traffic  control,  inventory 
management,  delays  at  toll  booths,  health  care  appointment  systems, 
machine  servicing,  water  resevoir  control,  airport  scheduling,  etc. 

In  an  extensive  bibliography,  Saaty  (1966)  claims  that  there  are  by  1966 
over  2000  references  in  queueing  theory,  and  comments  that  real  improve¬ 
ments  in  managing  congestion  phenomena  do  not  match  the  congestion 
caused  by  the  number  of  theoretical  papers  on  the  subject;  Lee  (1966) 
is  also  pessimistic.  Bhat  (1969)  refutes  these  arguments  in  a  con¬ 
venient  summary  of  the  field,  and  gives  a  more  selective  bibliography. 

In  the  prototypical  queueing  problem  we  imagine  that  customers 

1,2,3,  ...,  n,  ...  arrive  at  a  service  system  at  points  in  time 

t^,t^  +  t2,t^  +  t2  +  t^,  ...,  (tj^  +  t2  +  •••  t^),  ...  ,  and  queue  up  in 

front  of  a  single  server,  who  will  process  them  individually,  taking 

s^,S2,S2,  ...,  s^,  ...  units  of  time.  We  additionally  specify  FIFO 

(first-in,  first-out)  service  priority  and  assume  the  server  begins 

work  as  soon  as  a  customer  arrives.  If  we  let  w  be  the  waiting 

n 

time  in  the  queue  of  the  n*"^  customer  and  assume  that  the  first  customer 
arrives  when  the  server  is  idle,  we  find: 
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=  0 


(12.1) 


W2  =  max  (s^  -  t^jO) 


=  max  (W2  +  S2  -  t^jO) 


w  =  max  (w  +  s  -  t  ,0) 
n  n-1  n-i  n 


One  can  also  describe,  e.g.,  the  number  in  the  system  found  by  the 
t  h 

n  arrival.  Equations  similar  to  these  could,  in  principle,  be  found 
for  other  variations  in  service  priority,  if  the  number  of  servers 
were  increased  to  m  (in  parallel) ,  if  there  were  serial  stages  of 
queues  (with  or  without  intermediate  queues),  if  service  was  in  batches, 
etc.,  etc. 

The  typical  analytic  assumptions  about  the  input  and  service 
processes  are  that  the  arrival  spacings  {t^,t2,t2,  ...,  t^j  ...} 
are  independent  and  identically  distributed  random  variables  (thus 
arrivals  constitute  a  renewal  process) ,  and  service  times 
{s^,S2,S2,  ...,  s^,  ...}  are  also  i.i.d.r.v.s  ,  with  a  different 
distribution.  In  spite  of  the  simplicity  of  formulation  of  this 
so-called  "G/G/1"  queue,  only  a  few  general  results  are  known: 

(1)  Statistical  equilibrium  is  achieved  if  and  only  if  the  utilization 
factor,  p  =  s/t  (s,t  -  mean  service  and  inter-arrival  times), 

is  strictly  less  than  unity; 

(2)  The  fraction  of  time  the  server  is  idle  is  1  -  p  ; 

The  customer-average  mean  waiting  time  in  queue,  w  ,  and  the  time- 
average  mean  number  of  customers  in  the  queue,  q  ,  are  related  by: 

q  =  w/t  . 


(3) 
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Equivalents  of  these  results  hold  under  more  general  conditions,  for  example, 
if  there  are  m  parallel  servers,  or  if  a  different  priority  scheme  is  used. 
(3)  is  a  very  general  result  which  essentially  defines  what  we  mean  by  cus¬ 
tomer-average  wait  and  time-average  queue  (Little,  1961),  (Jewell,  1967), 
(Eilon,  1969),  (Maxwell,  1970).  Extensions  are  (Stidham,  1972),  (Brumelle, 


1972). 

Further  general  results  seem  very  difficult.  The  basic  problem  is  that 
til 

n*"  delay  content,  u  =  s  -  t,T,isa  two-sided  random  variable;  as  long 
■'  n  n  n+1 

as  the  partial  sums  u^  ,  u^  +  u^,  ...  are  positive,  they  are  identically 

W2,w^ .  However  (with  probability  one,  if  p  <  1)  ,  for  some  k  , 

Uf  +  U2  +  ...  +  will  be  negative,  =  0  ,  and  the  process  starts 

over.  The  analytic  determination  of  the  distribution  of  k  (the  number 
served  during  a  busy  period)  and  i  =  |u^  +  U2  +  ...  "*■  *^1  (the  length  of 
the  next  idle  period)  seems  very  difficult  in  the  general  case;  but,  if 
they  could  be  determined,  we  could  obtain  G/G/I  results,  such  as: 


9  2-2  2  — 

a  +  a  +  (t)  (1  -  p)  .2 

(12.2)  w  =  — - ^ - —  , 

2t(l  -  p)  2l 

2  2 

where  a  and  a  are  variance  of  the  inter-arrival  and  service  r.v.s, 
t  s 

—  2 

and  i  ,  i  the  first  two  moments  of  idle  time  (Marshall,  1968a).  Alter¬ 
natively,  we  must  find  the  distribution  of  w  from  a  Wiener-Hopf  integral 
equation. 

The  most  popular  historical  way  around  these  difficulties  has  been  to 
use  exponential  inter-arrival  (Poisson  arrival)  and  exponential  service 
distribution  assumptions.  Because  of  the  "memoryless"  properties  of  the 
exponential,  Pr  {x  >  x^  +  h  |  x  >  x^}  =  Pr  {x  >  h}  ,  every  interval  of 
time  is  a  regeneration  point,  and  queueing  systems  can  be  described  in  terms 
of  continuous-time  Markov  processes,  and  solved  by  linear  systems  of  first- 
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order  "birth-and-death"  differential  equations.  This  approach  enables  us 
to  model  unlimited  variations,  such  as  different  queue  disciplines  and 
priorities,  balking  and  reneging,  interrupted,  blocked,  and  controlled  ser¬ 
vice,  serial  and  parallel  stages,  etc.  Arbitrary  distributions  can  be  ap¬ 
proximated  through  sums  or  mixtures  of  random  variables.  Morse  (1958)  has 
many  useful  models  of  this  type. 

Kendall  (1951),  (1953)  was  the  first  to  show  that  only  one  of  the 

inter-arrival  or  service  distributions  need  be  exponential  to  complete  the 

analysis.  For  example,  in  the  case  of  Poisson  arrivals,  i  is  distributed 
2  -  2 

as  t  ,  =  (t)  ,  and  (12.2)  reduces  to  the  Pollaczek-Khintchlne  formula: 


(12.3) 


w 


2  -  2 
aj  +  (s)^ 

s 


2t(l  -  p) 


If  service  times  are  exponential,  one  can  analyze  an  imbedded  Markov  chain, 
even  with  m  servers;  see,  e.g.,  Kleinrock  (1967).  Unfortunately,  the  as¬ 
sumptions  of  exponentiality  led  naturally  to  the  use  of  transform  methods, 
and  the  papers  of  the  '60s  are  overburdened  with  the  machinery  of  LaPlace 
and  Fourier. 

Some  of  the  most  interesting  recent  research  in  queueing  theory  has 
been  in  the  area  of  approximate  and  bounding  results,  especially  to  the 
moments  of  the  waiting-time  distribution.  Kingman  (1962b)  showed  that 

(12.2)  has  the  strict  upper  bound 

2^2 
_  a  +  a 

(12.3)  w  _<  — ^ ^  =  W 

2t(l  -  p) 


for  all  GI/G/1  queues,  and  that  this  bound  is  a  good  approximation  for  w 
in  heavy  traffic  (p  1)  ,  when  w  is  then  approximately  exponentially 
distributed  (1962a)  (1962b). 
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Finding  a  strict  lower  bound  in  terms  of  moments  is  more  difficult. 
Marshall  (1968a)  makes  some  additional  shape  assumptions  on  the  distribu¬ 
tion  of  t  and  gets  sharp  lower  bounds.  For  example,  if  E{x  -  |  x  >  x^} 

is  decreasing  in  x^  (decreasing  mean  wait  for  the  next  customer  as  a  func¬ 
tion  of  clock  time  since  the  last  arrival) ,  he  obtains 


(12.4) 


W 

u 


which  bounds  q  to  within  one  customer!  Other  single-server  variations  are 
in  (Marshall,  1968b). 

Bounds  for  the  G/G/m  queue  are  given  by  Kingman  (1970)  and  Brumelle 
(1971),  and  have  been  improved  for  certain  special  cases;  a  convenient  sum¬ 
mary  is  in  (Kleinrock,  1976).  Finally,  we  should  mention  the  large  amount 
of  unpublished  work  on  approximations  for  single-  and  multi-server  Poisson 
queues  (Marchal,  Gross,  Harris,  1974),  (Nozaki  and  Ross,  1976),  (Takahashi, 
1976),  and  the  growing  interest  in  estimators  (Law,  1975). 

A  valid  criticism  of  all  the  above  models  is  that  they  are  only  useful 
in  stable  regimes;  time-varying  parameters  and  transient  response  are  diffi¬ 
cult  to  analyze,  except  in  the  simplest  systems.  However,  a  growing  litera¬ 
ture  in  queueing  theory  begins  by  approximating  the  arrival  and  departure 
processes  themselves,  making  first  a  deterministic  "fluid"  approximation  to 
the  average  values  of  these  processes,  and  then  adding  a  second-order  "dif¬ 
fusion"  approximation.  Gaver  (1968)  has  investigated  diffusion  approxima¬ 
tions  to  the  M/G/1  queue,  and  Newell  (1968),  (1971)  has  written  extensively 
on  "rush  hour"  traffic,  when  the  system  is  overloaded  for  a  period  of  time, 
and  then  recovers.  Although  the  concepts  are  simple,  the  analysis  leads  to 
Fokker-Planck  diffusion  equations,  and  requires  care  in  arguing  the  limiting 
approximations.  Kleinrock  (1976)  contains  a  clear  survey  of  diffusion  models; 
Whitt  (1974)  covers  recent  contributions  to  limit  theorems. 


46 


Perhaps  the  most  interesting  new  application  of  queueing  theory  has 
been  in  the  field  of  computer  time-sharing  systems.  Kleinrock  (1976)  con¬ 
tains  an  excellent  description  of  the  analytic  and  heuristic  models  developed 
to  analyze  multi-user  priority  schemes  and  design  computer  communication  net¬ 
works.  The  challenge  of  working  with  a  complicated  real  network  (AEIPANET) 
has  clearly  provided  a  fruitful  interaction  between  queueing  theory  and  prac¬ 
tice. 
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13.  RELIABILITY  THEORY 

The  early  models  of  reliability  theory  was  primarily  of  series-parallel 
connections  of  elements  with  exponential  lifetimes.  However,  since  the  early 
60' s  there  has  been  a  rapid  expansion  around  two  new  and  important  concepts. 
The  first  is  the  theory  of  coherent  structures,  which  provides  a  general 
framework  for  analyzing  systems  of  unreliable  elements.  The  second  idea 
is  to  assume  certain  monotone  shape  properties  of  the  lifetime  distributions 
(increasing  failure  rate  and  rate  average,  increasing  mean  residual  life, 

"new  better  than  used,"  etc.)  to  bound  complex  system  performance  measures, 
and  determine  optimal  replacement  policies  (Barlow  and  Proschan,  1965,  1975), 
(Proschan,  1976). 

Current  research  topics  are  surveyed  in  (Barlow  and  Proschan,  1976). 

Of  particular  interest  are  a  successful  multivariate  generalization  of  the 
failure  rate  (Marshall,  1975),  increased  interest  in  Bayesian  models  (Tsokos, 
1977),  and  the  use  of  fault  trees  to  systematically  develop  failure  modes  in 
complex  systems,  such  as  nuclear  reactors  (Barlow,  Fussell,  and  Singpurwalla, 


1975). 
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14.  FORECASTING  AND  BAYESIAN  STATISTICS 

There  are  two  developments  in  statistics  which  are  influencing  the 
methodology  of  operations  research.  In  the  empirical  forecasting  of  time 
series,  the  ARIMA  (auto-regressive  integrated  moving- average)  models  of  Box 
and  Jenkins  (1970)  provide  an  economical  framework  in  which  to  identify  rea¬ 
sonable  underlying  mechanisms  and  carry  out  the  necessary  computations.  The 
theory  also  puts  older  exponential-smoothing  heuristics  on  a  firm  basis. 

The  other  development  has  been  the  so-called  Bayesian  revolution.  Sup¬ 
pose  we  have  some  prior  information  about  a  random  parameter  0  (the  inputs 
or  control  settings  in  a  certain  process,  or  the  physical  or  economic  condi¬ 
tions  surrounding  a  certain  experiment,  or  the  skill  of  a  human  operator, 
etc.)  which  we  can  summarize  in  a  prior  density,  p(e)  ;  and  suppose,  for 
every  possible  value  of  0  ,  we  know  the  "likelihood",  p(x  |  0)  ,  the  con¬ 
ditional  density  of  observing  a  different  random  value,  x  ,  during  some  well- 
defined  experiment.  By  the  use  of  conditional  expectation  (Bayes'  Law),  we 
find  that,  posterior- to-observing  the  sample  value  x  =  x^  ,  we  can  redefine 
our  knowledge  about  0  to  obtain  the  posterior— to— data  density 

(14.1)  p(0  I  x^)  =  ^P(Xq  I  ®)  ‘PCQ)  . 

where  k  is  a  constant  to  normalize  p(0  |  x^)  .  The  current  controversy  in 
the  statistical  community  seems  to  stem  not  from  (14.1),  but  from  whether  a 
consulting  statistician  is  permitted  to  have  any  personal  beliefs  about  0 
to  include  in  the  prior,  p(0)  ,  or  whether  he  must  devise  methods  to  let 
the  data  somehow  "speak  for  itself"  (Savage,  1962)  (Barnett,  1973) . 

This  is  hardly  a  crisis  in  operation  research  systems  analysis  where 
the  ability  to  draw  on  prior  experience  and  analogous  situations  is  permitted, 
nay,  encouraged  in  estimation  procedures.  More  importantly,  the  Bayesian  ap¬ 
proach  reveals  paradoxes  in  the  classical  sampling— theory  school  of  statistics 
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(Lindley  1972,  1975),  (Basu,  1975),  and  in  spite  of  various  attempts  to  re¬ 
concile  the  two  approaches,  such  as  the  use  of  diffuse  priors  and  "empirical 
Bayes"  techniques,  it  seems  as  if  the  basic  sampling- theory  ideas,  such  as 
point  estimation,  significance  testing,  and  confidence  intervals,  must  be 
reformulated.  (Houle,  1973)  gives  about  2000  references  in  Bayesian  sta¬ 
tistics;  many  new  references  regularly  appear  in  O.R.  journals- 

Motivated  by  estimation  problems  in  insurance,  the  author  has  been  in¬ 
terested  in  Bayesian  prediction  schemes,  particularly  the  estimation  of  the 
mean  value  of  a  future  observation  posterior-to-data,  viz: 

(14.2)  E{x  I  x^}  =  J'J’xpix  1  6)p(0  I  x^)dxd6  , 

which  represents  the  "experience-rated  fair  premium"  in  insurance  terms. 

Actuaries  noticed  that,  for  many  priors  p(6)  and  likelyhoods  p(x  |  6)  , 

(14.2)  was  linear  in  the  data  x  ;  this  is  true  even  for  nonnormal  families, 

o 

and  the  general  conditions  under  which  this  is  true  are  now  known,  and  have 
been  extended  to  the  multi-dimensional  case  (Jewell,  1974). 

In  more  general  forecasting  and  regression  schemes,  the  Bayesian  mean 
may  not  be  linear  in  the  data;  however,  one  can  easily  find  the  best  linear 
approximation  through  the  use  of  least-squares  theory.  This  field  is  re¬ 
ferred  to  as  "credibility  theory"  in  the  actuarial  literature  (a  survey  is 
in  (Jewell,  1976)),  and  is  closely  allied  with  "linear  filter  theory"  in 
the  communications  field  (Sage  and  Melsa,  1971).  (Aitchison  and  Dunsmore, 
1975)  analyzes  other  Bayesian  prediction  schemes. 
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15.  INDUSTRIAL  AND  MANAGEMENT  MODELS 

This  section  will  briefly  survey  some  of  the  recent  trends  in  business 
models.  The  growth  and  influence  of  systems  ideas  over  the  last  twenty 
years  has  been  tremendous,  and  scientific  management  is  now  a  fact  of  life 
in  all  kinds  and  sizes  of  industries. 

Inventory  control  was  already  a  mature  field  by  the  60' s,  following  the 
development  of  the  basic  statistical  models  (Arrow,  Karlin,  Scarf,  1958) 
(Scarf,  Gilford,  Shelly,  1963) .  Since  that  time,  the  emphasis  has  been  on 
making  the  models  more  realistic  and  extensive.  (Gross  and  Schrady,  1976) 
is  a  recent  summary;  see  also  the  forthcoming  book  by  Silver  and  Peterson 
(1977).  (Muckstadt,  1973)  describes  a  large-scale  application. 

Turning  to  the  production  side,  an  important  new  trend  has  been  the  de¬ 
velopment  of  Integrated  systems  for  production  planning,  scheduling  and  in¬ 
ventory  control  (Hax  and  Golovin,  1974)  (Max  and  Mead,  1975)  (Hax, 1976b). 
(Bitran  and  Hax,  1976)  The  problems  of  logistics,  including  plant  location 
and  distribution,  have  also  been  the  subject  of  recent  intensive  investiga¬ 
tions  (Eilon,  Watson-Gandy,  Christof ides,  1971) .  (Francis  and  White,  1974) 

(Geisler, 1975)  (Geoffrion,  1975)  (Marlow,  1976)  See  especially  the  survey 
by  Hax  (1976a).  Multilevel  analysis  is  described  in  ( Jennergren, 1976) . 

Marketing,  on  the  other  hand,  is  an  area  which  only  recently  has  been 
quantified,  apparently  with  success.  Kotler  (1971)  provides  a  comprehensive 
survey;  recent  articles  of  interest  are  (Little, 1975)  and  (Hauser  and  Urban, 
1976). 

In  the  area  of  project  management,  the  most  important  development  of  the 
60' s  was  in  scheduling,  using  the  critical-path  methods  developed  for  the 
Polaris  missile  program  and  the  construction  of  the  S.S.  France.  The  basic 
models  belong  to  the  network  class  of  linear  programs,  have  very  simple 
algorithms,  and  are  now  routinely  used  in  all  major  construction  projects 
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(Thornley,  1968)  (Moder  and  Phillips,  1970)  (Lombaers,  1969) .  Since  that  time, 
attention  has  been  directed  towards  the  resource- loading  problem  -  an  inher¬ 
ently  difficult  problem  which  is  of  the  NP-complte  class  (Herroelen, 1972) 
is  a  convenient  survey.  (Shephard,  Al-Ayat,  and  Leachman,  1976)  is  a  differ¬ 
ent  modelling  approach,  using  dynamic  production  function  theory.  Another 
active  area  has  been  in  the  selection  and  budgeting  of  research  and  develop¬ 
ment  projects  (Gear,  Lockett  and  Pearson,  1971)  (Gear  and  Lockett, 1973) 

(Naslund  and  Sellstedt,  1974)  (Baker  and  Freeland,  1975) .  The  problems  of  de¬ 
tailed  manpower  scheduling  are  also  of  continued  interest  (Bennett  and  Potts, 
1968)  (Arabeyre  et  al,  1969);  Bodin  (1972)  gives  a  general  model.  (Baker,  1974) 
(Coffman,  1975)  use  recent  works  on  job/shop  scheduling  and  sequencing. 

Finally,  the  most  explosive  management  science  area  in  the  past  decade 
has  been  the  field  of  investment  and  finance.  Following  the  pioneering  idea 
of  Markowitz  (1959)  to  select  a  portfolio  of  investments  as  a  trade  off  be¬ 
tween  mean  return  and  variance,  many  different  extensions  have  been  made  in 
an  attempt  to  improve  investment  performance;  see  (Francis  and  Archer,  1971) 
(Sharpe, 1971)  (Lorie  and  Brealey,  1972) .  Part  of  the  problem  may  be  that  the 
stock  market  is  too  efficient  a  process  for  a  computer  to  make  money,  at  least 
in  the  long  run  (Fama,  1970)  (Granger  and  Morgenstern, 1970) .  The  optimal  de¬ 
sign  of  bond  maturity  schedules  is,  however,  a  more  tractable  problem  (Bradley 
and  Crane,  1975).  Money  managers  are  also  using  linear  programming  and  other 
methods  to  reduce  or  increase  "float"  (Caiman, 1968)  (Orgler,  1970)  (Orr,  1971). 

A  bibliography  of  3,600  works  in  the  finance  and  investment  area  is  in 
(Brealey  and  Pyle, 1973). 
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16.  THE  INFLUENCE  OF  O.R.  METHODOLOGIES 

Since  other  speakers  at  this  meeting  will  be  describing  various  appli¬ 
cations  of  operational  research/systems  analysis,  I  would  like  to  describe 
another  process  which  has  not  been  widely  noted — the  influence  of  O.R.  metho¬ 
dologies  upon  other  disciplines,  particularly  in  research  and  teaching. 

For  example,  we  are  so  used  to  talking  of  the  uses  of  mathematics,  it 
is  easy  to  overlook  the  stimulus  that  linear  programming  has  given  to  the 
study  of  convex  polytopes,  solutions  of  inequalities,  discrete  mathematics, 
and  graph  theory.  Linear  algebra,  including  a  brief  introduction  to  linear 
programming,  is  now  taught  to  all  engineering  freshmen  and  math  majors  at  my 
university;  simple  graphical-solution  linear  programs  even  appear  in  high 
school  "new  math"  courses.  New  algorithms,  based  upon  the  complementary  pivot 
theory  of  mathematical  programming,  give  promise  of  providing  practical  calcu¬ 
lation  of  fixed  points — an  achievement  which  will  find  wide  application  in 
both  pure  and  applied  mathematics  (Karamardian,  1976)  (Saigal,  1976) . 

Scarf  (1973)  has  already  applied  these  fixed-point  algorithms  to  the 
computation  of  economic  equilibria — a  difficult  problem  which  has  heretofore 
eluded  economists  for  even  modest-sized  problems.  Duality  theory,  with  its 
concepts  of  Imputed  values  of  resources  and  of  "pricing  out"  inefficient  ac¬ 
tivities,  has  proved  a  fertile  field  for  quantifying  basic  economic  notions 
such  as  marginal  costs.  Large-scale  economic  planning  and  optimization  is 
now  possible  on  a  scale  undreamed  of  twenty  years  ago,  thanks  to  modern 
linear  programming  codes.  Production  function  theory  has  changed  dramatic¬ 
ally  (Shephard,  1976) .  New  terms,  like  trade-off,  cost-benefit  analysis,  sub- 
optimization,  efficient  frontier,  and  decentralized  control  are  universally 
used  and  understood. 

In  statistics,  the  various  interesting  problems  posed  by  dynamic  pro¬ 
gramming,  decision  analysis,  Markov  programming,  etc.,  models  have  certainly 


53 


stimulated  research  in  statistical  decision  theory,  gambling  systems,  mar¬ 
tingales,  potential  theory,  limit  theorems  in  renewal  theory,  and  so  forth. 

The  concept  of  monotone  shape  characteristics  for  distributions,  introduced 
in  reliability  applications,  has  provided  a  new  approach  to  bounding  mo¬ 
ments  in  random  walks,  And,  the  many  possible  variations  in  queueing  models 
has  provided  a  torrent  of  marginal  contributions  to  the  statistical  journals, 
much  to  the  concern  of  the  editors  (Pyke, 1975). 

I  have  already  described  the  changes  in  almost  every  field  of  business 
administration;  the  same  can  be  said  about  industrial  engineering.  The 
methodologies  have  also  been  adopted  by  electrical  engineering,  especially 
in  control  theory  and  in  communications  network  design,  as  described  earlier. 
Transportation  engineering  relies  heavily  upon  queueing  theory,  network  flows, 
dynamic  programming,  etc.  (Gazis,  1976) .  Critical  path  scheduling  is  taught 
routinely  in  construction  engineering.  Statistical  models  of  wear  are  useful 
in  metal  behavior  studies.  Dynamic  programming  is  used  for  nuclear  fuel 
management,  and  fault  tree  analysis  to  isolate  nuclear  reactor  shutdown  se¬ 
quences.  And  so  on. 

A  variety  of  new  sister  disciplines  have  sprung  up  which  also  use 
O.R.  methodologies,  as  a  glance  at  the  new  journals  will  reveal:  urban  plan¬ 
ning,  environmental  engineering,  energy  analysis,  health  care  systems,  etc. 

And  especially  in  computer  science  it  is  possible  to  trace  the  Influence 
of  O.R.  methodologies:  from  queueing  theory  for  the  design  of  computer  sys¬ 
tems,  through  graph  theory  and  combinatorics  for  the  design  of  efficient  data 
structures  and  manipulation  procedures,  to  the  common  concerns  for  develop¬ 
ing,  testing,  and  implementing  efficient  algorithms  (Aho,  Hopcraft,  and 
Ullman, 1975) .  It  seems  to  me  that  there  is  a  certain  amount  of  tension  just 
now  between  O.R.  and  this  newest  engineering  science  discipline,  caused  in 
part  by  the  shift  in  popularity  and  research  support,  but  also  by  the  reali¬ 
zation  that  problems  of  algorithmic  efficiency  have  become  too  esoteric  for 
the  O.R.  analyst,  and  require  the  attention  of  a  different  kind  of  specialist. 
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17.  CRISES  WITHIN  THE  PROFESSION 

This  tremendous  activity  and  expansion  in  the  field  of  operational 
research/systems  analysis  has,  however,  been  achieved  at  the  cost  of 
considerable  disorder  within  the  profession,  as  the  discussion  sections 
of  the  journals  and  the  conference  roundtable  discussions  reveal;  see  the 
references  in  Klein  and  Butkovitch  (1976). 

The  first  crisis  is  over  the  incredible  proliferation  in  papers  and 
specialty  journals.  ORSA/TIMS  conference  dimensions  are  staggering,  as 
are  the  numbers  of  regular  meetings  of  numerous  special-interest  groups 
and  new  specialty  societies.  Kendall  (1960)  estimates  that,  in  1958,  one 
would  have  to  scan  five  journals  to  cover  1/3  of  the  English-language 
contributions  and  about  18  journals  to  cover  1/2  of  the  literature;  my 
estimate  of  the  current  situation,  based  upon  scanning  our  university 
libraries,  is  that  about  15  and  50  journals,  respectively,  would  be  needed 
in  1976.  Even  traditional  journals  have  fissioned  into  several  parts. 

One  wonders  what  libraries  can  afford  to  stock  them  all,  or  how  many  people 
have  "xerox  subscriptions."  Some  will  say  that  this  proliferation  is  the 
direct  result  of  the  "publish  or  perish"  promotion  criteria  of  American 
universities;  others  point  to  regulations  requiring  one  to  present  a  paper 
to  secure  travel  support  to  a  technical  meeting.  But  it  is  clear  that  this 
communication  explosion  is  affecting  other  sciences  as  well  and  there  is 
no  easy  solution  in  sight. 

With  this  proliferation  has  come  increasingly  narrow  specialization, 
which  we  see  in  academia  when  students  insist  they  want  to  major  in  mathe¬ 
matical  programming  or  queueing  theory,  and  the  faculty  advisors  permit  them 
to  do  so.  Klein  and  Butkovitch  (1976)  suggests  darkly  that  this  is  a  natural 
phenomenon,  since  the  OR/MS  academic  discipline  is  an  institutionalized 
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system  of  exchange  which  sets  up  modular  specializations  in  order  to  insure 
its  own  survival;  they  see  little  hope  for  institutional  change. 

Another  crisis,  perhaps  more  pertinent  to  the  U.S.A.  than  to  the  U.K., 
has  been  the  apprent  separation  between  theory  and  application.  Practi¬ 
tioners  regularly  rage  at  the  mathematical  "overkill"  in  the  pages  of  the 
journals,  and  yearn  for  the  good  old  days  when  a  simple  model  could 
explicate  an  observed  phenomenon;  in  rebuttal,  researchers  point  to  the 
trivial  level  of  many  of  the  applications  papers  ("How  I  ..."),  and  the 
universal  lack  of  sponsorship  by  industry  of  meaningful  research  programs. 

There  are  continuing  criticisms  of  the  academic  programs  (Schrady, 
1976)  which  imply  that  OR/MS  training  is  not  responsive  to  the  needs  of 
industry  -  being  too  technique-oriented,  over-specialized,  having  little 
understanding  of  the  total  systems  approach,  unable  to  collect  and  organize 
data  or  write  management  reports,  and  so  forth. 

Putting  these  tensions  all  together,  adding  the  success  of  new  fields, 
such  as  computer  science,  and  contemplating  the  rapid  rate  of  adoption  of 
0.  R.  methodologies  by  the  applications  fields  (business;  economics;  trans¬ 
portation,  environmental,  and  communications  engineering,  etc.),  has  led 
to  a  larger  malady,  which  might  be  called  a  crisis  of  confidence.  We  see 
this  in  the  searching  self-examination  of  many  of  the  roundtable  dis¬ 
cussions  ("Is  Mathematical  Programming  Moribund?",  "Are  We  Gambling  on 
OR/MS  Education?"),  and  in  statements  to  the  effect  that  operational 
research  has  promised  too  much,  delivered  too  little,  and  should  now  be 
given  a  decent  burial. 
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18.  FUTURE  PROSPECTS 

I  prefer  to  take  a  somewhat  more  balanced  view  of  these  crises.  For 
example,  there  is  evidence  that,  in  the  U.S.,  the  societies  are  moving  to 
correct  some  of  the  earlier  excesses.  ORSA  and  TIMS  have  been  growing  more 
closely  together,  running  simultaneous  conferences,  and  sharing  membership 
administration  facilities.  Their  publication  policies  have  also  been 
coordinated  and  rationalized:  A  new  journal,  called  Mathematics  of 
Operations  Research,  has  been  established  to  attract  important  theoretical 
^i^ticles .  Interfaces  has  been  designated  as  the  new  joint  medium  for 
describing  operational  problems  of  implementing  or  using  OR/MS;  the  quality 
of  articles  is  improving  under  the  new  editor,  who  insists  that  all 
^^ticles  be  readable  and  that  equations  be  relegated  to  the  appendices. 

The  parent  journals.  Operations  Research  and  Management  Science,  are  now 
free  to  concentrate  on  major  articles  of  interest  to  all  members  of  the 
profession. 

Another  interesting  development  has  been  the  sponsorship  of  a  prize 
competition  for  papers  on  successful  applications  of  MS  by  the 
TIMS  College  on  Practice  (Interfaces.  Vol.  6,  No.  1).  The  rules  are  strict: 
the  entries  must  report  a  completed,  practical  application  and  must  present 
results  that  have  had  a  significant  impact  on  the  performance  of  the 
organization  under  study,  as  certified  by  management.  Because  practitioners 
do  not  normally  publish  such  studies,  the  prize  is  set  at  a  significant  level 
($6,000  for  the  1977  competition).  The  actual  presentations  and  the  written 
papers  are  extremely  interesting,  in  my  opinion,  and  provide  a  standard  of 
professional  practice  previously  unavailable. 
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There  are  no  easy  solutions  to  the  publication  explosion  problem, 
although  de-emphasis  of  published  works  as  a  university  promotion  criterion 
would  certainly  help,  as  would  resolution  of  some  of  the  legal  and  institu¬ 
tional  problems  surrounding  the  inexpensive  duplication  and  distribution  of 
papers.  1  believe  in  market  mechanisms  and  the  freedom  to  fail;  journals 
which  do  not  serve  some  useful  purpose  will  soon  vanish  from  the  scene. 

Who  is  to  question  the  utility  of  those  which  survive? 

Academic  programs  are  easily  criticized,  but  one  must  remember  that 
there  were  few  texts  or  courses  before  1960,  and  new  programs  had  to  be 
grafted  onto  a  variety  of  different  educational  formats.  It  is  true  there 
has  been  a  great  deal  of  theoretical  activity  relative  to  the  actual  appli¬ 
cations,  but  this  is  the  characteristic  of  "normal  science"  (Kuhn,  1970), 
whose  first  priority  is  to  structure  the  appropriate  and  potentially  useful 
knowledge  and  explore  its  theoretical  facets.  "Few  people  who  are  not 
actually  practitioners  of  a  mature  science  realize  how  much  mop-up  work  of 
this  sort  a  paradigm  leaves  to  be  done  or  quite  how  fascinating  such  work 
can  prove  in  the  execution"  (op.  cit.,  p.  24). 

It  is  also  true  that  our  recent  graduates,  now  staffing  industry, 
government  and  other  teaching  faculties,  have  over-emphasized  technique  in 
place  of  application,  and  did  not  participate  in  the  same  school  of  hard 
knocks  and  simple  models  that  reared  our  founders.  But  the  same  can  be 
said  of  any  profession.  These  young  people  are  extremely  bright,  and,  I 
believe,  more  adaptable  to  new  demands  by  society  than  many  of  the  tired 
pioneers.  As  far  as  teaching  the  systems  approach  is  concerned,  philosophy 
is  fine  (Churchman,  1968) ,  but  what  is  needed  are  more  excellent  texts  like 
(White,  1975),  and  good  professional  articles,  developing  ideas  like  those 
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in  (Bishop,  1972)  and  (Liebman,  1976) .  More  consistent  signals  from  research 
funding  agencies  would  help  useful  academic  development,  as  would  more 
interest  by  industry  in  providing  research  problems  and  support,  and  helping 
educational  programs  to  make  closer  ties  with  reality. 

OR  academicians,  on  the  other  hand,  must  learn  to  let  go  of  any 
proprietary  feelings  they  have  about  the  methodologies  they  helped  to 
develop,  and  pay  closer  attention  to  the  substantial  issues  facing  their 
field  of  primary  interest,  be  it  business,  government,  or  industrial 
engineering.  The  great  strength  of  the  profession  has  come  from  the 
ability  to  construct  interesting  models  of  real-world  phenomenon,  and  to 
use  the  solutions  to  resolve  actual  problems.  The  outlines  of  the  applied 
methods  are  now  clear  for  all  to  see,  and,  at  some  point,  methodology 
becomes  pure  mathematics  or  statistics  or  computer  science,  the  concern 
of  other  specialists.  It  is  a  mark  of  maturity  that  our  methods  are  now 
influencing  other  fields,  and  that  the  availability  of  these  new  support 
skills  frees  us  to  return  to  the  central  issues  of  modelling  and  problem¬ 


solving. 
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